[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:07 dist-test-slave-vxj2 krb5kdc[28874](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:07 dist-test-slave-vxj2 krb5kdc[28874](info): set up 2 sockets
Aug 11 22:09:07 dist-test-slave-vxj2 krb5kdc[28874](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260811 22:09:08.878700 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.169s user 0.006s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:09:08 dist-test-slave-vxj2 krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:09:09.026846 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:44685
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:41783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:44685
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:09.171818 28889 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:09.172190 28889 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:09.172242 28889 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:09.177492 28889 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:09.177587 28889 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:09.177608 28889 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:09.177626 28889 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:09.177642 28889 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:09.184942 28889 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:41783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:44685
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:44685
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28889
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:09.186729 28889 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:09.188254 28889 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:09.203661 28901 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:09.204336 28889 server_base.cc:1034] running on GCE node
W20260811 22:09:09.204408 28899 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:09.204663 28895 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:09.205709 28889 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:09.207849 28889 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:09.211248 28889 hybrid_clock.cc:648] HybridClock initialized: now 1786486149211180 us; error 72 us; skew 500 ppm
Aug 11 22:09:09 dist-test-slave-vxj2 krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486149, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:09.215324 28889 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:09:09.216780 28889 webserver.cc:460] Webserver started at http://127.28.47.190:36449/ using document root <none> and password file <none>
I20260811 22:09:09.217895 28889 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:09.218008 28889 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:09.218456 28889 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:09.221618 28889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "c607bc53749b4b7391b61ffa744a6ac9"
format_stamp: "Formatted at 2026-08-11 22:09:09 on dist-test-slave-vxj2"
server_key: "44d86314fe00db3fc7b0afbd183df0c0"
server_key_iv: "2eabd02f572dd40bc1057a6596687ade"
server_key_version: "encryptionkey@0"
I20260811 22:09:09.222241 28889 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "c607bc53749b4b7391b61ffa744a6ac9"
format_stamp: "Formatted at 2026-08-11 22:09:09 on dist-test-slave-vxj2"
server_key: "44d86314fe00db3fc7b0afbd183df0c0"
server_key_iv: "2eabd02f572dd40bc1057a6596687ade"
server_key_version: "encryptionkey@0"
I20260811 22:09:09.228329 28889 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:09:09.232656 28910 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:09.234716 28889 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.002s
I20260811 22:09:09.234987 28889 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "c607bc53749b4b7391b61ffa744a6ac9"
format_stamp: "Formatted at 2026-08-11 22:09:09 on dist-test-slave-vxj2"
server_key: "44d86314fe00db3fc7b0afbd183df0c0"
server_key_iv: "2eabd02f572dd40bc1057a6596687ade"
server_key_version: "encryptionkey@0"
I20260811 22:09:09.235147 28889 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:09.256480 28889 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:09.260339 28889 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:09.260643 28889 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:09.273542 28889 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:44685
I20260811 22:09:09.273603 28967 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:44685 every 8 connection(s)
I20260811 22:09:09.275131 28889 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:09.277216 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 28889
I20260811 22:09:09.277424 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:09.279093 28862 external_mini_cluster.cc:1428] Setting key 6ef2493ed42af115ed9a85973217daea
I20260811 22:09:09.281018 28968 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:09.293690 28968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9: Bootstrap starting.
I20260811 22:09:09.298728 28968 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:09.300388 28968 log.cc:824] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:09.304602 28968 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9: No bootstrap required, opened a new log
Aug 11 22:09:09 dist-test-slave-vxj2 krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486148, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:09.310285 28968 raft_consensus.cc:348] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c607bc53749b4b7391b61ffa744a6ac9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 44685 } }
I20260811 22:09:09.310686 28968 raft_consensus.cc:374] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:09.310833 28968 raft_consensus.cc:729] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c607bc53749b4b7391b61ffa744a6ac9, State: Initialized, Role: FOLLOWER
I20260811 22:09:09.311829 28968 consensus_queue.cc:260] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c607bc53749b4b7391b61ffa744a6ac9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 44685 } }
I20260811 22:09:09.312111 28968 raft_consensus.cc:388] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:09.312290 28968 raft_consensus.cc:482] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:09.312463 28968 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:09.314154 28968 raft_consensus.cc:504] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c607bc53749b4b7391b61ffa744a6ac9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 44685 } }
I20260811 22:09:09.314826 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:09.285777 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47508 (local address 127.28.47.190:44685)
0811 22:09:09.286494 (+ 717us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:09.286513 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:09.286789 (+ 276us) server_negotiation.cc:407] Connection header received
0811 22:09:09.291876 (+ 5087us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:09.291897 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:09.292397 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:09.292726 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:09.296991 (+ 4265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:09.298242 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:09.300419 (+ 2177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:09.301268 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:09.301507 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:09.307624 (+ 6117us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:09.307666 (+ 42us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:09.307681 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:09.307722 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:09.309823 (+ 2101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:09.310509 (+ 686us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:09.310514 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:09.310519 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:09.310651 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:09.311273 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:09.311276 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:09.311278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:09.311891 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:09.312170 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:09.313123 (+ 953us) server_negotiation.cc:299] Negotiation successful
0811 22:09:09.313483 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":267,"threads_started":1}
I20260811 22:09:09.314908 28968 leader_election.cc:302] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c607bc53749b4b7391b61ffa744a6ac9; no voters:
I20260811 22:09:09.315429 28968 leader_election.cc:288] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:09.315622 28973 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:09.315871 28973 raft_consensus.cc:686] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [term 1 LEADER]: Becoming Leader. State: Replica: c607bc53749b4b7391b61ffa744a6ac9, State: Running, Role: LEADER
I20260811 22:09:09.316421 28973 consensus_queue.cc:237] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c607bc53749b4b7391b61ffa744a6ac9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 44685 } }
I20260811 22:09:09.317622 28968 sys_catalog.cc:564] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:09.319749 28977 sys_catalog.cc:455] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c607bc53749b4b7391b61ffa744a6ac9. Latest consensus state: current_term: 1 leader_uuid: "c607bc53749b4b7391b61ffa744a6ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c607bc53749b4b7391b61ffa744a6ac9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 44685 } } }
I20260811 22:09:09.319921 28977 sys_catalog.cc:458] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:09.320333 28974 sys_catalog.cc:455] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c607bc53749b4b7391b61ffa744a6ac9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c607bc53749b4b7391b61ffa744a6ac9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 44685 } } }
I20260811 22:09:09.320461 28974 sys_catalog.cc:458] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9 [sys.catalog]: This master's current role is: LEADER
W20260811 22:09:09.326205 28991 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:09:09.326356 28991 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:09:09.326463 28989 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:09.327433 28989 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:09.335492 28989 catalog_manager.cc:1347] Generated new cluster ID: 5072a285a0d34c97bfef3aec92067e14
I20260811 22:09:09.335597 28989 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:09.347347 28989 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:09.348449 28989 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:09.361464 28989 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c607bc53749b4b7391b61ffa744a6ac9: Generated new TSK 0
I20260811 22:09:09.362543 28989 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:09:09.421672 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:44685
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:41783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:09.603286 28997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:09.603624 28997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:09.603674 28997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:09.608289 28997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:09.608384 28997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:09.608451 28997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:09.614758 28997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:41783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:44685
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:09.616137 28997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:09.617403 28997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:09:09.630263 28997 server_base.cc:1034] running on GCE node
W20260811 22:09:09.630600 29005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:09.630551 29002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:09.630628 29003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:09.631212 28997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:09.631947 28997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:09.633206 28997 hybrid_clock.cc:648] HybridClock initialized: now 1786486149633138 us; error 35 us; skew 500 ppm
Aug 11 22:09:09 dist-test-slave-vxj2 krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486149, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:09.637053 28997 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:09:09.638140 28997 webserver.cc:460] Webserver started at http://127.28.47.129:43981/ using document root <none> and password file <none>
I20260811 22:09:09.638792 28997 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:09.638897 28997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:09.639201 28997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:09.641901 28997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "d57faabde8d6481a98185a27ddf3bf0a"
format_stamp: "Formatted at 2026-08-11 22:09:09 on dist-test-slave-vxj2"
server_key: "4a6eaeed9d60d314e86438d57adc315d"
server_key_iv: "bd718ed83cdb7928f9af5be38d690f5a"
server_key_version: "encryptionkey@0"
I20260811 22:09:09.642647 28997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "d57faabde8d6481a98185a27ddf3bf0a"
format_stamp: "Formatted at 2026-08-11 22:09:09 on dist-test-slave-vxj2"
server_key: "4a6eaeed9d60d314e86438d57adc315d"
server_key_iv: "bd718ed83cdb7928f9af5be38d690f5a"
server_key_version: "encryptionkey@0"
I20260811 22:09:09.648157 28997 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.003s
I20260811 22:09:09.651441 29013 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:09.653223 28997 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:09.653409 28997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "d57faabde8d6481a98185a27ddf3bf0a"
format_stamp: "Formatted at 2026-08-11 22:09:09 on dist-test-slave-vxj2"
server_key: "4a6eaeed9d60d314e86438d57adc315d"
server_key_iv: "bd718ed83cdb7928f9af5be38d690f5a"
server_key_version: "encryptionkey@0"
I20260811 22:09:09.653556 28997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:09.670624 28997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:09.674842 28997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:09.675191 28997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:09.676173 28997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:09.677698 28997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:09.677786 28997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:09.677929 28997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:09.677978 28997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:09.700759 28997 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:40429
I20260811 22:09:09.700839 29149 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:40429 every 8 connection(s)
I20260811 22:09:09.701946 28997 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:09.703678 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 28997
I20260811 22:09:09.703894 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:09:09.704312 28862 external_mini_cluster.cc:1428] Setting key 604484c7b74af93ec24e12ff50f61b77
Aug 11 22:09:09 dist-test-slave-vxj2 krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486149, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:09:09.722426 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:09.704716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:60329 (local address 127.28.47.190:44685)
0811 22:09:09.704875 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:09.704880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:09.706766 (+ 1886us) server_negotiation.cc:407] Connection header received
0811 22:09:09.708776 (+ 2010us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:09.708791 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:09.708912 (+ 121us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:09.709108 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:09.710852 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:09.711761 (+ 909us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:09.713505 (+ 1744us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:09.713988 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:09.714136 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:09.717548 (+ 3412us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:09.717570 (+ 22us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:09.717575 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:09.717612 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:09.719552 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:09.720448 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:09.720451 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:09.720452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:09.720539 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:09.720945 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:09.720949 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:09.720950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:09.721228 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:09.721342 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:09.722069 (+ 727us) server_negotiation.cc:299] Negotiation successful
0811 22:09:09.722232 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:09:09.723500 29152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:09.705283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:44685 (local address 127.28.47.129:60329)
0811 22:09:09.706574 (+ 1291us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:09.706642 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:09.708304 (+ 1662us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:09.709439 (+ 1135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:09.709447 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:09.709929 (+ 482us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:09.710657 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:09.710676 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:09.711942 (+ 1266us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:09.711945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:09.713349 (+ 1404us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:09.713361 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:09.714184 (+ 823us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:09.714187 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:09.714440 (+ 253us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:09.715321 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:09.715339 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:09.717252 (+ 1913us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:09.719758 (+ 2506us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:09.719765 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:09.719787 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:09.720308 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:09.720651 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:09.720654 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:09.720656 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:09.720831 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:09.721341 (+ 510us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:09.721350 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:09.721707 (+ 357us) client_negotiation.cc:769] Sending connection context
0811 22:09:09.721981 (+ 274us) client_negotiation.cc:240] Negotiation successful
0811 22:09:09.722383 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":948,"thread_start_us":272,"threads_started":1}
I20260811 22:09:09.725356 29150 heartbeater.cc:347] Connected to a master server at 127.28.47.190:44685
I20260811 22:09:09.725699 29150 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:09.726320 29150 heartbeater.cc:511] Master 127.28.47.190:44685 requested a full tablet report, sending...
I20260811 22:09:09.728667 28930 ts_manager.cc:194] Registered new tserver with Master: d57faabde8d6481a98185a27ddf3bf0a (127.28.47.129:40429)
I20260811 22:09:09.730312 28930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:60329
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:09:09.745662 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:44685
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:41783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:09.920168 29155 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:09.920518 29155 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:09.920570 29155 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:09.925429 29155 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:09.925510 29155 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:09.925575 29155 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:09.931578 29155 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:41783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:44685
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29155
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:09.932797 29155 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:09.933976 29155 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:09.945798 29166 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:09.946112 29155 server_base.cc:1034] running on GCE node
W20260811 22:09:09.945915 29164 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:09.945958 29163 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:09.946815 29155 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:09.947572 29155 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:09.948755 29155 hybrid_clock.cc:648] HybridClock initialized: now 1786486149948739 us; error 31 us; skew 500 ppm
Aug 11 22:09:09 dist-test-slave-vxj2 krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486149, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:09.952705 29155 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:09:09.953915 29155 webserver.cc:460] Webserver started at http://127.28.47.130:43145/ using document root <none> and password file <none>
I20260811 22:09:09.954571 29155 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:09.954643 29155 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:09.954890 29155 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:09.957468 29155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "6494cd07174e41ab80537ed154905308"
format_stamp: "Formatted at 2026-08-11 22:09:09 on dist-test-slave-vxj2"
server_key: "133d5116ed1dfcc1a40a3b0dd5def6ba"
server_key_iv: "6837f752eaf697d3dd857a87d489c339"
server_key_version: "encryptionkey@0"
I20260811 22:09:09.958187 29155 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "6494cd07174e41ab80537ed154905308"
format_stamp: "Formatted at 2026-08-11 22:09:09 on dist-test-slave-vxj2"
server_key: "133d5116ed1dfcc1a40a3b0dd5def6ba"
server_key_iv: "6837f752eaf697d3dd857a87d489c339"
server_key_version: "encryptionkey@0"
I20260811 22:09:09.963146 29155 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:09:09.966352 29174 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:09.968050 29155 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:09.968225 29155 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "6494cd07174e41ab80537ed154905308"
format_stamp: "Formatted at 2026-08-11 22:09:09 on dist-test-slave-vxj2"
server_key: "133d5116ed1dfcc1a40a3b0dd5def6ba"
server_key_iv: "6837f752eaf697d3dd857a87d489c339"
server_key_version: "encryptionkey@0"
I20260811 22:09:09.968387 29155 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:09.991932 29155 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:09.996152 29155 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:09.996482 29155 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:09.997488 29155 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:09.998565 29155 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:09.998644 29155 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:09.998715 29155 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:09.998759 29155 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:10.025019 29155 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:42127
I20260811 22:09:10.025144 29303 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:42127 every 8 connection(s)
I20260811 22:09:10.026784 29155 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:10.037006 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 29155
I20260811 22:09:10.037125 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:09:10.037416 28862 external_mini_cluster.cc:1428] Setting key 39177b3cc737d6eb8e201127fff4dc90
Aug 11 22:09:10 dist-test-slave-vxj2 krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486149, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:10.046072 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:10.029269 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:57651 (local address 127.28.47.190:44685)
0811 22:09:10.029437 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:10.029442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:10.031069 (+ 1627us) server_negotiation.cc:407] Connection header received
0811 22:09:10.032316 (+ 1247us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:10.032320 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:10.032378 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:10.032471 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:10.034246 (+ 1775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:10.034808 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:10.036347 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:10.036596 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:10.036831 (+ 235us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:10.040416 (+ 3585us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:10.040419 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:10.040421 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:10.040449 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:10.042793 (+ 2344us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:10.043755 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:10.043760 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:10.043761 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:10.043868 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:10.044418 (+ 550us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:10.044421 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:10.044423 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:10.044705 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:10.044819 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:10.045752 (+ 933us) server_negotiation.cc:299] Negotiation successful
0811 22:09:10.045930 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:09:10.047569 29306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:10.029773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:44685 (local address 127.28.47.130:57651)
0811 22:09:10.030869 (+ 1096us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:10.030923 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:10.032054 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:10.032695 (+ 641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:10.032703 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:10.033367 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:10.034070 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:10.034088 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:10.034996 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:10.034999 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:10.036199 (+ 1200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:10.036211 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:10.036810 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:10.036813 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:10.037113 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:10.038046 (+ 933us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:10.038070 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:10.040206 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:10.042985 (+ 2779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:10.042994 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:10.043024 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:10.043516 (+ 492us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:10.043992 (+ 476us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:10.043998 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:10.044002 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:10.044252 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:10.044852 (+ 600us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:10.044863 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:10.045368 (+ 505us) client_negotiation.cc:769] Sending connection context
0811 22:09:10.045632 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 22:09:10.046100 (+ 468us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":829,"thread_start_us":216,"threads_started":1}
I20260811 22:09:10.050320 29304 heartbeater.cc:347] Connected to a master server at 127.28.47.190:44685
I20260811 22:09:10.051203 29304 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:10.052466 29304 heartbeater.cc:511] Master 127.28.47.190:44685 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:09:10.054462 28930 ts_manager.cc:194] Registered new tserver with Master: 6494cd07174e41ab80537ed154905308 (127.28.47.130:42127)
I20260811 22:09:10.055279 28930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:57651
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:09:10.081938 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:44685
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:41783
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:10.259263 29309 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:10.259598 29309 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:10.259646 29309 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:10.264170 29309 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:10.264253 29309 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:10.264317 29309 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:10.270833 29309 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:41783
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:44685
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29309
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:10.272147 29309 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:10.273437 29309 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:10.285871 29315 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:10.286029 29309 server_base.cc:1034] running on GCE node
W20260811 22:09:10.285845 29318 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:10.286278 29314 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:10.286769 29309 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:10.287618 29309 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:10.288807 29309 hybrid_clock.cc:648] HybridClock initialized: now 1786486150288774 us; error 58 us; skew 500 ppm
Aug 11 22:09:10 dist-test-slave-vxj2 krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486150, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:10.292359 29309 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:09:10.293486 29309 webserver.cc:460] Webserver started at http://127.28.47.131:43073/ using document root <none> and password file <none>
I20260811 22:09:10.294095 29309 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:10.294162 29309 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:10.294340 29309 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:10.296715 29309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "91df7c7c015849bd8fff04c47e15c69f"
format_stamp: "Formatted at 2026-08-11 22:09:10 on dist-test-slave-vxj2"
server_key: "ddd52fb11db7074525202eb4fe738e2f"
server_key_iv: "c6a073b094bb29af179b959f24a57d38"
server_key_version: "encryptionkey@0"
I20260811 22:09:10.297405 29309 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "91df7c7c015849bd8fff04c47e15c69f"
format_stamp: "Formatted at 2026-08-11 22:09:10 on dist-test-slave-vxj2"
server_key: "ddd52fb11db7074525202eb4fe738e2f"
server_key_iv: "c6a073b094bb29af179b959f24a57d38"
server_key_version: "encryptionkey@0"
I20260811 22:09:10.301708 29309 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:09:10.304554 29325 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:10.306156 29309 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:10.306308 29309 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "91df7c7c015849bd8fff04c47e15c69f"
format_stamp: "Formatted at 2026-08-11 22:09:10 on dist-test-slave-vxj2"
server_key: "ddd52fb11db7074525202eb4fe738e2f"
server_key_iv: "c6a073b094bb29af179b959f24a57d38"
server_key_version: "encryptionkey@0"
I20260811 22:09:10.306460 29309 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:10.323477 29309 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:10.327746 29309 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:10.328241 29309 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:10.329293 29309 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:10.330423 29309 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:10.330513 29309 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:10.330598 29309 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:10.330648 29309 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:10.353567 29309 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:38135
I20260811 22:09:10.353674 29461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:38135 every 8 connection(s)
I20260811 22:09:10.354738 29309 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:10.362288 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 29309
I20260811 22:09:10.362412 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:09:10.362706 28862 external_mini_cluster.cc:1428] Setting key f7ff059b379d2d6f0f0a049ed459a405
Aug 11 22:09:10 dist-test-slave-vxj2 krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486150, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:10.372090 28971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:10.357061 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:44117 (local address 127.28.47.190:44685)
0811 22:09:10.357224 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:10.357228 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:10.358925 (+ 1697us) server_negotiation.cc:407] Connection header received
0811 22:09:10.360176 (+ 1251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:10.360180 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:10.360247 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:10.360362 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:10.362087 (+ 1725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:10.362810 (+ 723us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:10.364575 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:10.364829 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:10.364951 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:10.368027 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:10.368030 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:10.368032 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:10.368061 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:10.369466 (+ 1405us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:10.370043 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:10.370055 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:10.370057 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:10.370152 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:10.370617 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:10.370620 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:10.370622 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:10.370836 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:10.370925 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:10.371582 (+ 657us) server_negotiation.cc:299] Negotiation successful
0811 22:09:10.371883 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:09:10.372877 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:10.357573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:44685 (local address 127.28.47.131:44117)
0811 22:09:10.358729 (+ 1156us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:10.358786 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:10.359941 (+ 1155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:10.360556 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:10.360564 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:10.361072 (+ 508us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:10.361807 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:10.361824 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:10.362966 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:10.362969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:10.364444 (+ 1475us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:10.364455 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:10.365000 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:10.365004 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:10.365325 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:10.366144 (+ 819us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:10.366162 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:10.367850 (+ 1688us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:10.369581 (+ 1731us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:10.369587 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:10.369601 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:10.369894 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:10.370275 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:10.370278 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:10.370279 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:10.370451 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:10.370940 (+ 489us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:10.370945 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:10.371224 (+ 279us) client_negotiation.cc:769] Sending connection context
0811 22:09:10.371421 (+ 197us) client_negotiation.cc:240] Negotiation successful
0811 22:09:10.371735 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":835,"thread_start_us":245,"threads_started":1}
I20260811 22:09:10.374500 29462 heartbeater.cc:347] Connected to a master server at 127.28.47.190:44685
I20260811 22:09:10.374794 29462 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:10.375422 29462 heartbeater.cc:511] Master 127.28.47.190:44685 requested a full tablet report, sending...
I20260811 22:09:10.377091 28930 ts_manager.cc:194] Registered new tserver with Master: 91df7c7c015849bd8fff04c47e15c69f (127.28.47.131:38135)
I20260811 22:09:10.377904 28930 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:44117
I20260811 22:09:10.378260 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:09:10 dist-test-slave-vxj2 krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486150, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:09:10 dist-test-slave-vxj2 krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486150, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.47.129@KRBTEST.COM
I20260811 22:09:10.413376 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:10.400027 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54048 (local address 127.28.47.129:40429)
0811 22:09:10.400365 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:10.400375 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:10.400536 (+ 161us) server_negotiation.cc:407] Connection header received
0811 22:09:10.400743 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:10.400750 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:10.400965 (+ 215us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:10.401133 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:10.402061 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:10.403011 (+ 950us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:10.404152 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:10.404684 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:10.404842 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:10.408504 (+ 3662us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:10.408521 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:10.408539 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:10.408578 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:10.410943 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:10.411512 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:10.411537 (+ 25us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:10.411539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:10.411636 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:10.412132 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:10.412135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:10.412136 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:10.412446 (+ 310us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:10.412626 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:10.412951 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 22:09:10.413102 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":90,"threads_started":1}
W20260811 22:09:10.414598 29052 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:54048
Aug 11 22:09:10 dist-test-slave-vxj2 krb5kdc[28874](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:09:10 dist-test-slave-vxj2 krb5kdc[28874](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486150, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.129@KRBTEST.COM
I20260811 22:09:10.445509 29469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:10.436729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54052 (local address 127.28.47.129:40429)
0811 22:09:10.436916 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:10.436920 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:10.436944 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:09:10.437097 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:10.437099 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:10.437146 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:10.437274 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:10.438024 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:10.438582 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:10.439499 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:10.439761 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:10.439845 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:10.442440 (+ 2595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:10.442444 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:10.442446 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:10.442472 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:10.443752 (+ 1280us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:10.444184 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:10.444186 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:10.444188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:10.444268 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:10.444525 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:10.444527 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:10.444528 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:10.444700 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:10.444806 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:10.445121 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 22:09:10.445310 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:09:10.446801 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 28997
I20260811 22:09:10.457599 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 29155
I20260811 22:09:10.467344 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 29309
I20260811 22:09:10.476313 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 28889
2026-08-11T22:09:10Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2807 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:10 dist-test-slave-vxj2 krb5kdc[29476](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:10 dist-test-slave-vxj2 krb5kdc[29476](info): set up 2 sockets
Aug 11 22:09:10 dist-test-slave-vxj2 krb5kdc[29476](info): commencing operation
krb5kdc: starting...
W20260811 22:09:11.694662 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.192s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:09:11 dist-test-slave-vxj2 krb5kdc[29476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:09:11.850776 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:46539
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:36467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:46539
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:11.979308 29492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:11.979712 29492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:11.979801 29492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:11.984117 29492 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:11.984272 29492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:11.984318 29492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:11.984360 29492 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:11.984441 29492 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:11.990455 29492 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:36467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:46539
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:46539
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:11.991798 29492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:11.992997 29492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:09:12.002744 29492 server_base.cc:1034] running on GCE node
W20260811 22:09:12.002667 29502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:12.002696 29500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:12.002668 29499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:12.003713 29492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:12.005033 29492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:12.006285 29492 hybrid_clock.cc:648] HybridClock initialized: now 1786486152006268 us; error 33 us; skew 500 ppm
Aug 11 22:09:12 dist-test-slave-vxj2 krb5kdc[29476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:12.009859 29492 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:09:12.011505 29492 webserver.cc:460] Webserver started at http://127.28.47.190:35889/ using document root <none> and password file <none>
I20260811 22:09:12.012405 29492 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:12.012528 29492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:12.012792 29492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:12.015467 29492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "419619aa5d3d40d594e0db3e3e854429"
format_stamp: "Formatted at 2026-08-11 22:09:12 on dist-test-slave-vxj2"
server_key: "b6272a811113def507204693b37f0642"
server_key_iv: "00f7b8cdede0616b7309c917d4eba543"
server_key_version: "encryptionkey@0"
I20260811 22:09:12.016090 29492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "419619aa5d3d40d594e0db3e3e854429"
format_stamp: "Formatted at 2026-08-11 22:09:12 on dist-test-slave-vxj2"
server_key: "b6272a811113def507204693b37f0642"
server_key_iv: "00f7b8cdede0616b7309c917d4eba543"
server_key_version: "encryptionkey@0"
I20260811 22:09:12.020676 29492 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:09:12.023733 29514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:12.025553 29492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:12.025707 29492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "419619aa5d3d40d594e0db3e3e854429"
format_stamp: "Formatted at 2026-08-11 22:09:12 on dist-test-slave-vxj2"
server_key: "b6272a811113def507204693b37f0642"
server_key_iv: "00f7b8cdede0616b7309c917d4eba543"
server_key_version: "encryptionkey@0"
I20260811 22:09:12.025851 29492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:12.049350 29492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:12.053516 29492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:12.053716 29492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:12.062470 29492 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:46539
I20260811 22:09:12.062484 29580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:46539 every 8 connection(s)
I20260811 22:09:12.063627 29492 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:12.067173 29581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:12.070199 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 29492
I20260811 22:09:12.070346 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:12.070647 28862 external_mini_cluster.cc:1428] Setting key 9c0d00ab3b39f4df2d0a6cb999552c68
I20260811 22:09:12.074764 29581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429: Bootstrap starting.
I20260811 22:09:12.077576 29581 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:12.078503 29581 log.cc:824] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:09:12 dist-test-slave-vxj2 krb5kdc[29476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:12.081666 29581 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429: No bootstrap required, opened a new log
I20260811 22:09:12.088040 29581 raft_consensus.cc:348] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419619aa5d3d40d594e0db3e3e854429" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46539 } }
I20260811 22:09:12.088392 29581 raft_consensus.cc:374] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:12.088451 29581 raft_consensus.cc:729] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 419619aa5d3d40d594e0db3e3e854429, State: Initialized, Role: FOLLOWER
I20260811 22:09:12.088893 29585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.072347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38436 (local address 127.28.47.190:46539)
0811 22:09:12.072976 (+ 629us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:12.072994 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:12.073037 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:09:12.073964 (+ 927us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:12.073987 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:12.074404 (+ 417us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:12.074825 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:12.075794 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.076983 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.078110 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.078696 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.078917 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.082305 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:12.082313 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:12.082334 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:12.082374 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:12.084844 (+ 2470us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:12.085501 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:12.085509 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:12.085516 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:12.085688 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:12.086023 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:12.086026 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:12.086029 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:12.086708 (+ 679us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:12.086955 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:12.087233 (+ 278us) server_negotiation.cc:299] Negotiation successful
0811 22:09:12.087613 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":452,"thread_start_us":174,"threads_started":1}
I20260811 22:09:12.089039 29581 consensus_queue.cc:260] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419619aa5d3d40d594e0db3e3e854429" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46539 } }
I20260811 22:09:12.089247 29581 raft_consensus.cc:388] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:12.089320 29581 raft_consensus.cc:482] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:12.089416 29581 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:12.090737 29581 raft_consensus.cc:504] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419619aa5d3d40d594e0db3e3e854429" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46539 } }
I20260811 22:09:12.091176 29581 leader_election.cc:302] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 419619aa5d3d40d594e0db3e3e854429; no voters:
I20260811 22:09:12.091609 29581 leader_election.cc:288] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:12.091738 29587 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:12.092027 29587 raft_consensus.cc:686] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [term 1 LEADER]: Becoming Leader. State: Replica: 419619aa5d3d40d594e0db3e3e854429, State: Running, Role: LEADER
I20260811 22:09:12.092483 29587 consensus_queue.cc:237] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419619aa5d3d40d594e0db3e3e854429" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46539 } }
I20260811 22:09:12.093534 29581 sys_catalog.cc:564] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:12.094647 29592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "419619aa5d3d40d594e0db3e3e854429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419619aa5d3d40d594e0db3e3e854429" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46539 } } }
I20260811 22:09:12.094715 29588 sys_catalog.cc:455] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 419619aa5d3d40d594e0db3e3e854429. Latest consensus state: current_term: 1 leader_uuid: "419619aa5d3d40d594e0db3e3e854429" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "419619aa5d3d40d594e0db3e3e854429" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46539 } } }
I20260811 22:09:12.094827 29592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:12.094836 29588 sys_catalog.cc:458] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:12.095428 29600 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:12.099673 29600 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:12.105942 29600 catalog_manager.cc:1347] Generated new cluster ID: a4e78fb007b7463e885780fac8293d97
I20260811 22:09:12.106045 29600 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:12.118333 29600 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:12.119498 29600 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:12.126475 29600 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 419619aa5d3d40d594e0db3e3e854429: Generated new TSK 0
I20260811 22:09:12.127348 29600 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:09:12.170686 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46539
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:36467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:12.305606 29613 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:12.306036 29613 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:12.306137 29613 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:12.310667 29613 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:12.310911 29613 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:12.311887 29613 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:12.318421 29613 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:36467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46539
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29613
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:12.319852 29613 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:12.321069 29613 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:12.332520 29622 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:12.332522 29620 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:12.332542 29618 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:12.333099 29613 server_base.cc:1034] running on GCE node
I20260811 22:09:12.333647 29613 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:12.334398 29613 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:12.335657 29613 hybrid_clock.cc:648] HybridClock initialized: now 1786486152335615 us; error 50 us; skew 500 ppm
Aug 11 22:09:12 dist-test-slave-vxj2 krb5kdc[29476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:12.339331 29613 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:09:12.340466 29613 webserver.cc:460] Webserver started at http://127.28.47.129:41441/ using document root <none> and password file <none>
I20260811 22:09:12.341117 29613 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:12.341260 29613 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:12.341535 29613 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:12.343888 29613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "5cf37584e09b4a6db362f1eeeb18bfd3"
format_stamp: "Formatted at 2026-08-11 22:09:12 on dist-test-slave-vxj2"
server_key: "f908407173e88e073f81230669cdf9f9"
server_key_iv: "1eb0b8d18e57d755962c1f2af574c242"
server_key_version: "encryptionkey@0"
I20260811 22:09:12.344558 29613 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "5cf37584e09b4a6db362f1eeeb18bfd3"
format_stamp: "Formatted at 2026-08-11 22:09:12 on dist-test-slave-vxj2"
server_key: "f908407173e88e073f81230669cdf9f9"
server_key_iv: "1eb0b8d18e57d755962c1f2af574c242"
server_key_version: "encryptionkey@0"
I20260811 22:09:12.349090 29613 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:09:12.352180 29631 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:12.353907 29613 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:12.354102 29613 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "5cf37584e09b4a6db362f1eeeb18bfd3"
format_stamp: "Formatted at 2026-08-11 22:09:12 on dist-test-slave-vxj2"
server_key: "f908407173e88e073f81230669cdf9f9"
server_key_iv: "1eb0b8d18e57d755962c1f2af574c242"
server_key_version: "encryptionkey@0"
I20260811 22:09:12.354257 29613 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:12.370838 29613 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:12.375219 29613 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:12.375479 29613 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:12.376238 29613 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:12.377362 29613 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:12.377485 29613 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:12.377575 29613 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:12.377612 29613 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:12.388653 29613 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:46345
I20260811 22:09:12.388723 29764 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:46345 every 8 connection(s)
I20260811 22:09:12.389847 29613 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:12.390594 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 29613
I20260811 22:09:12.390722 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:09:12.391031 28862 external_mini_cluster.cc:1428] Setting key d3226a5b59c2a42d15ab092c43e7d3d3
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
Aug 11 22:09:12 dist-test-slave-vxj2 krb5kdc[29476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:12.414810 29585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.392185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:41985 (local address 127.28.47.190:46539)
0811 22:09:12.392328 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:12.392333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:12.393831 (+ 1498us) server_negotiation.cc:407] Connection header received
0811 22:09:12.395447 (+ 1616us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:12.395453 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:12.395529 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:12.395632 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:12.397285 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.398214 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.400151 (+ 1937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.400682 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.400833 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.408537 (+ 7704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:12.408541 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:12.408572 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:12.408622 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:12.411469 (+ 2847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:12.412305 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:12.412312 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:12.412318 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:12.412464 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:12.413069 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:12.413075 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:12.413077 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:12.413467 (+ 390us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:12.413616 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:12.414305 (+ 689us) server_negotiation.cc:299] Negotiation successful
0811 22:09:12.414522 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:09:12.415767 29768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.392783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:46539 (local address 127.28.47.129:41985)
0811 22:09:12.393614 (+ 831us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:12.393681 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:12.395145 (+ 1464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:12.396028 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:12.396037 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:12.396460 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:12.397038 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:12.397054 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.398431 (+ 1377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.398435 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:12.399971 (+ 1536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:12.399987 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.401066 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.401076 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:12.401356 (+ 280us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.402331 (+ 975us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:12.402351 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:12.408225 (+ 5874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:12.411703 (+ 3478us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:12.411710 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:12.411729 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:12.412165 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:12.412595 (+ 430us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:12.412597 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:12.412599 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:12.412938 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:12.413634 (+ 696us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:12.413640 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:12.413986 (+ 346us) client_negotiation.cc:769] Sending connection context
0811 22:09:12.414194 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 22:09:12.414531 (+ 337us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":221,"threads_started":1}
I20260811 22:09:12.418116 29765 heartbeater.cc:347] Connected to a master server at 127.28.47.190:46539
I20260811 22:09:12.418460 29765 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:12.419343 29765 heartbeater.cc:511] Master 127.28.47.190:46539 requested a full tablet report, sending...
I20260811 22:09:12.421756 29540 ts_manager.cc:194] Registered new tserver with Master: 5cf37584e09b4a6db362f1eeeb18bfd3 (127.28.47.129:46345)
I20260811 22:09:12.423557 29540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:41985
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:09:12.432024 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46539
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:36467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:12.563228 29771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:12.563575 29771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:12.563628 29771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:12.568100 29771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:12.568197 29771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:12.568262 29771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:12.574514 29771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:36467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46539
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:12.575922 29771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:12.577132 29771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:12.588757 29778 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:12.588725 29779 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:12.588985 29771 server_base.cc:1034] running on GCE node
W20260811 22:09:12.588661 29782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:12.589632 29771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:12.590438 29771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:12.591678 29771 hybrid_clock.cc:648] HybridClock initialized: now 1786486152591640 us; error 50 us; skew 500 ppm
Aug 11 22:09:12 dist-test-slave-vxj2 krb5kdc[29476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:12.595407 29771 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:09:12.596778 29771 webserver.cc:460] Webserver started at http://127.28.47.130:36107/ using document root <none> and password file <none>
I20260811 22:09:12.597594 29771 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:12.597713 29771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:12.598023 29771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:12.600723 29771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "dad1febb1e324cf49491f20fdfe13af2"
format_stamp: "Formatted at 2026-08-11 22:09:12 on dist-test-slave-vxj2"
server_key: "a07b53f1202e57cd8b21d049314274ad"
server_key_iv: "82df6f0baaf3fa6a42ca4558c14c1014"
server_key_version: "encryptionkey@0"
I20260811 22:09:12.601497 29771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "dad1febb1e324cf49491f20fdfe13af2"
format_stamp: "Formatted at 2026-08-11 22:09:12 on dist-test-slave-vxj2"
server_key: "a07b53f1202e57cd8b21d049314274ad"
server_key_iv: "82df6f0baaf3fa6a42ca4558c14c1014"
server_key_version: "encryptionkey@0"
I20260811 22:09:12.606772 29771 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.002s
I20260811 22:09:12.609910 29791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:12.611479 29771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:12.611663 29771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "dad1febb1e324cf49491f20fdfe13af2"
format_stamp: "Formatted at 2026-08-11 22:09:12 on dist-test-slave-vxj2"
server_key: "a07b53f1202e57cd8b21d049314274ad"
server_key_iv: "82df6f0baaf3fa6a42ca4558c14c1014"
server_key_version: "encryptionkey@0"
I20260811 22:09:12.611822 29771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:12.631410 29771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:12.635078 29771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:12.635314 29771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:12.635994 29771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:12.636940 29771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:12.637015 29771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:12.637090 29771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:12.637144 29771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:12.647046 29771 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:37161
I20260811 22:09:12.647068 29925 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:37161 every 8 connection(s)
I20260811 22:09:12.648095 29771 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:12.652091 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 29771
I20260811 22:09:12.652237 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:09:12.652655 28862 external_mini_cluster.cc:1428] Setting key 8a5179db0a047de7a10bfa631b685e87
Aug 11 22:09:12 dist-test-slave-vxj2 krb5kdc[29476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:09:12.667385 29585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.650440 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:49371 (local address 127.28.47.190:46539)
0811 22:09:12.650654 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:12.650658 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:12.651642 (+ 984us) server_negotiation.cc:407] Connection header received
0811 22:09:12.653752 (+ 2110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:12.653755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:12.653805 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:12.653875 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:12.655643 (+ 1768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.656207 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.657695 (+ 1488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.658087 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.658187 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.661930 (+ 3743us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:12.661938 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:12.661945 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:12.661995 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:12.664139 (+ 2144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:12.664910 (+ 771us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:12.664917 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:12.664920 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:12.665045 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:12.665665 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:12.665672 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:12.665676 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:12.666064 (+ 388us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:12.666214 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:12.667083 (+ 869us) server_negotiation.cc:299] Negotiation successful
0811 22:09:12.667220 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":108}
I20260811 22:09:12.668337 29928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.650873 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:46539 (local address 127.28.47.130:49371)
0811 22:09:12.651455 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:12.651504 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:12.653483 (+ 1979us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:12.654098 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:12.654107 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:12.654636 (+ 529us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:12.655445 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:12.655472 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.656369 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.656376 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:12.657543 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:12.657556 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.658328 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.658335 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:12.658610 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.659410 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:12.659439 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:12.661620 (+ 2181us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:12.664360 (+ 2740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:12.664367 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:12.664385 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:12.664734 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:12.665287 (+ 553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:12.665290 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:12.665292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:12.665481 (+ 189us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:12.666253 (+ 772us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:12.666260 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:12.666625 (+ 365us) client_negotiation.cc:769] Sending connection context
0811 22:09:12.666835 (+ 210us) client_negotiation.cc:240] Negotiation successful
0811 22:09:12.667273 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":369,"thread_start_us":157,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
I20260811 22:09:12.670014 29926 heartbeater.cc:347] Connected to a master server at 127.28.47.190:46539
I20260811 22:09:12.670408 29926 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:12.671178 29926 heartbeater.cc:511] Master 127.28.47.190:46539 requested a full tablet report, sending...
I20260811 22:09:12.672708 29540 ts_manager.cc:194] Registered new tserver with Master: dad1febb1e324cf49491f20fdfe13af2 (127.28.47.130:37161)
I20260811 22:09:12.673499 29540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:49371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:09:12.693928 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46539
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:36467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:12.827472 29931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:12.827827 29931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:12.827880 29931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:12.832649 29931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:12.832759 29931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:12.832830 29931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:12.838850 29931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:36467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46539
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:12.840238 29931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:12.841502 29931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:12.853313 29940 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:12.853493 29931 server_base.cc:1034] running on GCE node
W20260811 22:09:12.853322 29943 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:12.853327 29941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:12.854198 29931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:12.855016 29931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:12.856241 29931 hybrid_clock.cc:648] HybridClock initialized: now 1786486152856228 us; error 40 us; skew 500 ppm
Aug 11 22:09:12 dist-test-slave-vxj2 krb5kdc[29476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:12.864436 29931 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:09:12.865650 29931 webserver.cc:460] Webserver started at http://127.28.47.131:42823/ using document root <none> and password file <none>
I20260811 22:09:12.866482 29931 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:12.866600 29931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:12.866864 29931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:12.869352 29931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "876ab7d7852f4548b3d595e8ec8fb280"
format_stamp: "Formatted at 2026-08-11 22:09:12 on dist-test-slave-vxj2"
server_key: "fbac5f205d0e9fe4b3352f7338c8ccd6"
server_key_iv: "db7ee6d7b3c365fe9a09e01d376e60da"
server_key_version: "encryptionkey@0"
I20260811 22:09:12.870016 29931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "876ab7d7852f4548b3d595e8ec8fb280"
format_stamp: "Formatted at 2026-08-11 22:09:12 on dist-test-slave-vxj2"
server_key: "fbac5f205d0e9fe4b3352f7338c8ccd6"
server_key_iv: "db7ee6d7b3c365fe9a09e01d376e60da"
server_key_version: "encryptionkey@0"
I20260811 22:09:12.874462 29931 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:09:12.877542 29952 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:12.879235 29931 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:12.879427 29931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "876ab7d7852f4548b3d595e8ec8fb280"
format_stamp: "Formatted at 2026-08-11 22:09:12 on dist-test-slave-vxj2"
server_key: "fbac5f205d0e9fe4b3352f7338c8ccd6"
server_key_iv: "db7ee6d7b3c365fe9a09e01d376e60da"
server_key_version: "encryptionkey@0"
I20260811 22:09:12.879580 29931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:12.898762 29931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:12.903092 29931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:12.903378 29931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:12.904090 29931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:12.905128 29931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:12.905288 29931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:12.905385 29931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:12.905432 29931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:12.916229 29931 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:35835
I20260811 22:09:12.916266 30083 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:35835 every 8 connection(s)
I20260811 22:09:12.917505 29931 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:12.923864 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 29931
I20260811 22:09:12.924011 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:09:12.924271 28862 external_mini_cluster.cc:1428] Setting key d186750a7724b5ce991f055912e2e6fc
Aug 11 22:09:12 dist-test-slave-vxj2 krb5kdc[29476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486152, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:12.933919 29585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.919904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:48657 (local address 127.28.47.190:46539)
0811 22:09:12.920081 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:12.920084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:12.921135 (+ 1051us) server_negotiation.cc:407] Connection header received
0811 22:09:12.922147 (+ 1012us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:12.922150 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:12.922212 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:12.922310 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:12.923690 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.924481 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.926002 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.926242 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.926340 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.929518 (+ 3178us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:12.929521 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:12.929523 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:12.929553 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:12.931122 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:12.931753 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:12.931764 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:12.931766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:12.931856 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:12.932252 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:12.932256 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:12.932258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:12.932497 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:12.932744 (+ 247us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:12.933536 (+ 792us) server_negotiation.cc:299] Negotiation successful
0811 22:09:12.933714 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:09:12.934900 30086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.920326 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:46539 (local address 127.28.47.131:48657)
0811 22:09:12.920939 (+ 613us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:12.920989 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:12.921917 (+ 928us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:12.922493 (+ 576us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:12.922500 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:12.922931 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:12.923470 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:12.923484 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.924644 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.924650 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:12.925861 (+ 1211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:12.925872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.926412 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.926416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:12.926646 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.927428 (+ 782us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:12.927446 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:12.929309 (+ 1863us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:12.931302 (+ 1993us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:12.931316 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:12.931330 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:12.931622 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:12.931965 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:12.931969 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:12.931971 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:12.932141 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:12.932746 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:12.932756 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:12.933229 (+ 473us) client_negotiation.cc:769] Sending connection context
0811 22:09:12.933464 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 22:09:12.933884 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":164,"threads_started":1}
I20260811 22:09:12.936360 30084 heartbeater.cc:347] Connected to a master server at 127.28.47.190:46539
I20260811 22:09:12.936676 30084 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:12.937414 30084 heartbeater.cc:511] Master 127.28.47.190:46539 requested a full tablet report, sending...
I20260811 22:09:12.938696 29540 ts_manager.cc:194] Registered new tserver with Master: 876ab7d7852f4548b3d595e8ec8fb280 (127.28.47.131:35835)
I20260811 22:09:12.939247 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:09:12.939427 29540 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:48657
I20260811 22:09:12.954012 29585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.944685 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38446 (local address 127.28.47.190:46539)
0811 22:09:12.944894 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:12.944900 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:12.945132 (+ 232us) server_negotiation.cc:407] Connection header received
0811 22:09:12.945367 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:12.945371 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:12.945440 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:12.945541 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:12.946422 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.947027 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.948025 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.948450 (+ 425us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.948597 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.950350 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:12.950354 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:12.950356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:12.950410 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:12.951949 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:12.952450 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:12.952453 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:12.952454 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:12.952542 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:12.952856 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:12.952859 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:12.952861 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:12.953291 (+ 430us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:12.953422 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:12.953736 (+ 314us) server_negotiation.cc:299] Negotiation successful
0811 22:09:12.953877 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 22:09:12.962479 29540 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:09:12.966238 29540 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:12.984970 30097 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.978138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:37161 (local address 127.0.0.1:45170)
0811 22:09:12.979117 (+ 979us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:12.979131 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:12.979241 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:12.979898 (+ 657us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:12.979904 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:12.979930 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:12.980085 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:12.980093 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.981356 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.981362 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:12.983052 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:12.983064 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.984099 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.984103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:12.984347 (+ 244us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.984427 (+ 80us) client_negotiation.cc:769] Sending connection context
0811 22:09:12.984552 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:09:12.984750 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":907,"mutex_wait_us":7,"thread_start_us":132,"threads_started":1}
I20260811 22:09:12.984988 30096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.978127 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:35835 (local address 127.0.0.1:44218)
0811 22:09:12.978504 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:12.978560 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:12.978695 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:12.980178 (+ 1483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:12.980182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:12.980203 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:12.980316 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:12.980324 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.981860 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.981863 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:12.983127 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:12.983141 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.984424 (+ 1283us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.984428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:12.984543 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.984571 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:09:12.984619 (+ 48us) client_negotiation.cc:240] Negotiation successful
0811 22:09:12.984767 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":198,"mutex_wait_us":57,"thread_start_us":110,"threads_started":1}
I20260811 22:09:12.985421 30101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.979055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44218 (local address 127.28.47.131:35835)
0811 22:09:12.979718 (+ 663us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:12.979730 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:12.979756 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:09:12.979844 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:12.979849 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:12.980019 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:12.980200 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:12.980446 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.981685 (+ 1239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.983320 (+ 1635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.984271 (+ 951us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.984878 (+ 607us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.985041 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:12.985110 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 22:09:12.985252 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":556,"thread_start_us":134,"threads_started":1}
I20260811 22:09:12.985683 30100 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.978931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:46345 (local address 127.0.0.1:58756)
0811 22:09:12.979497 (+ 566us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:12.979513 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:12.979603 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:12.980368 (+ 765us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:12.980371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:12.980389 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:12.980466 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:12.980474 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.981904 (+ 1430us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.981907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:12.983795 (+ 1888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:12.983808 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.985240 (+ 1432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.985244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:12.985337 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.985367 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:09:12.985442 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:09:12.985553 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":487,"mutex_wait_us":709,"thread_start_us":101,"threads_started":1}
I20260811 22:09:12.986004 30098 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.978285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45170 (local address 127.28.47.130:37161)
0811 22:09:12.978968 (+ 683us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:12.978982 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:12.979355 (+ 373us) server_negotiation.cc:407] Connection header received
0811 22:09:12.979446 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:12.979452 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:12.979689 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:12.979877 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:12.980249 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.981210 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.983210 (+ 2000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.983949 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.984076 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.984354 (+ 278us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:12.985718 (+ 1364us) server_negotiation.cc:299] Negotiation successful
0811 22:09:12.985840 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":572,"thread_start_us":119,"threads_started":1}
I20260811 22:09:12.986294 30102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:12.979383 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58756 (local address 127.28.47.129:46345)
0811 22:09:12.979770 (+ 387us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:12.979775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:12.979795 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:09:12.979902 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:12.979907 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:12.980088 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:12.980225 (+ 137us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:12.980610 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.981786 (+ 1176us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.983965 (+ 2179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:12.985007 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:12.985169 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:12.985991 (+ 822us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:12.986051 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 22:09:12.986151 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":98,"threads_started":1}
I20260811 22:09:12.989221 29849 tablet_service.cc:1467] Processing CreateTablet for tablet 9d971a64a1784c0a84ed725a6babab03 (DEFAULT_TABLE table=test-table [id=0948c22a77c04d1db651e5bcc7831a06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:12.989323 30008 tablet_service.cc:1467] Processing CreateTablet for tablet 9d971a64a1784c0a84ed725a6babab03 (DEFAULT_TABLE table=test-table [id=0948c22a77c04d1db651e5bcc7831a06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:12.989320 29688 tablet_service.cc:1467] Processing CreateTablet for tablet 9d971a64a1784c0a84ed725a6babab03 (DEFAULT_TABLE table=test-table [id=0948c22a77c04d1db651e5bcc7831a06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:12.990869 29849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d971a64a1784c0a84ed725a6babab03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:12.992259 29688 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d971a64a1784c0a84ed725a6babab03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:12.992820 30008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9d971a64a1784c0a84ed725a6babab03. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:12.999784 30103 tablet_bootstrap.cc:492] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2: Bootstrap starting.
I20260811 22:09:13.003450 30104 tablet_bootstrap.cc:492] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3: Bootstrap starting.
I20260811 22:09:13.004498 30103 tablet_bootstrap.cc:654] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:13.004819 30107 tablet_bootstrap.cc:492] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280: Bootstrap starting.
I20260811 22:09:13.006218 30103 log.cc:824] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:13.006935 30104 tablet_bootstrap.cc:654] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:13.007442 30107 tablet_bootstrap.cc:654] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:13.008122 30104 log.cc:824] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:13.008865 30107 log.cc:824] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:13.011070 30107 tablet_bootstrap.cc:492] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280: No bootstrap required, opened a new log
I20260811 22:09:13.011150 30104 tablet_bootstrap.cc:492] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3: No bootstrap required, opened a new log
I20260811 22:09:13.011339 30107 ts_tablet_manager.cc:1397] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:09:13.011361 30104 ts_tablet_manager.cc:1397] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 22:09:13.015520 30103 tablet_bootstrap.cc:492] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2: No bootstrap required, opened a new log
I20260811 22:09:13.016013 30103 ts_tablet_manager.cc:1397] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2: Time spent bootstrapping tablet: real 0.017s user 0.002s sys 0.007s
I20260811 22:09:13.017879 30104 raft_consensus.cc:348] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad1febb1e324cf49491f20fdfe13af2" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37161 } } peers { permanent_uuid: "876ab7d7852f4548b3d595e8ec8fb280" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35835 } } peers { permanent_uuid: "5cf37584e09b4a6db362f1eeeb18bfd3" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 46345 } }
I20260811 22:09:13.018216 30104 raft_consensus.cc:374] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:13.018273 30104 raft_consensus.cc:729] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5cf37584e09b4a6db362f1eeeb18bfd3, State: Initialized, Role: FOLLOWER
I20260811 22:09:13.019151 30107 raft_consensus.cc:348] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad1febb1e324cf49491f20fdfe13af2" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37161 } } peers { permanent_uuid: "876ab7d7852f4548b3d595e8ec8fb280" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35835 } } peers { permanent_uuid: "5cf37584e09b4a6db362f1eeeb18bfd3" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 46345 } }
I20260811 22:09:13.019516 30104 consensus_queue.cc:260] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad1febb1e324cf49491f20fdfe13af2" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37161 } } peers { permanent_uuid: "876ab7d7852f4548b3d595e8ec8fb280" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35835 } } peers { permanent_uuid: "5cf37584e09b4a6db362f1eeeb18bfd3" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 46345 } }
I20260811 22:09:13.019641 30107 raft_consensus.cc:374] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:13.019707 30107 raft_consensus.cc:729] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 876ab7d7852f4548b3d595e8ec8fb280, State: Initialized, Role: FOLLOWER
I20260811 22:09:13.020363 30107 consensus_queue.cc:260] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad1febb1e324cf49491f20fdfe13af2" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37161 } } peers { permanent_uuid: "876ab7d7852f4548b3d595e8ec8fb280" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35835 } } peers { permanent_uuid: "5cf37584e09b4a6db362f1eeeb18bfd3" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 46345 } }
I20260811 22:09:13.020869 30104 ts_tablet_manager.cc:1428] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3: Time spent starting tablet: real 0.009s user 0.004s sys 0.005s
I20260811 22:09:13.021505 29765 heartbeater.cc:503] Master 127.28.47.190:46539 was elected leader, sending a full tablet report...
I20260811 22:09:13.021695 30107 ts_tablet_manager.cc:1428] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280: Time spent starting tablet: real 0.010s user 0.007s sys 0.004s
I20260811 22:09:13.022113 30084 heartbeater.cc:503] Master 127.28.47.190:46539 was elected leader, sending a full tablet report...
I20260811 22:09:13.025245 30103 raft_consensus.cc:348] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad1febb1e324cf49491f20fdfe13af2" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37161 } } peers { permanent_uuid: "876ab7d7852f4548b3d595e8ec8fb280" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35835 } } peers { permanent_uuid: "5cf37584e09b4a6db362f1eeeb18bfd3" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 46345 } }
I20260811 22:09:13.025775 30103 raft_consensus.cc:374] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:13.025892 30103 raft_consensus.cc:729] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dad1febb1e324cf49491f20fdfe13af2, State: Initialized, Role: FOLLOWER
I20260811 22:09:13.026508 30103 consensus_queue.cc:260] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad1febb1e324cf49491f20fdfe13af2" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37161 } } peers { permanent_uuid: "876ab7d7852f4548b3d595e8ec8fb280" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35835 } } peers { permanent_uuid: "5cf37584e09b4a6db362f1eeeb18bfd3" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 46345 } }
I20260811 22:09:13.027961 30103 ts_tablet_manager.cc:1428] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2: Time spent starting tablet: real 0.012s user 0.009s sys 0.000s
I20260811 22:09:13.028434 29926 heartbeater.cc:503] Master 127.28.47.190:46539 was elected leader, sending a full tablet report...
W20260811 22:09:13.141621 29767 tablet.cc:2369] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:09:13.149629 29927 tablet.cc:2369] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:09:13.168924 30085 tablet.cc:2369] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:09:13.170758 30113 raft_consensus.cc:482] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:13.171032 30113 raft_consensus.cc:504] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad1febb1e324cf49491f20fdfe13af2" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37161 } } peers { permanent_uuid: "876ab7d7852f4548b3d595e8ec8fb280" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35835 } } peers { permanent_uuid: "5cf37584e09b4a6db362f1eeeb18bfd3" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 46345 } }
I20260811 22:09:13.172904 30113 leader_election.cc:288] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 876ab7d7852f4548b3d595e8ec8fb280 (127.28.47.131:35835), 5cf37584e09b4a6db362f1eeeb18bfd3 (127.28.47.129:46345)
I20260811 22:09:13.178274 30114 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:13.173150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:35835 (local address 127.28.47.130:44447)
0811 22:09:13.173542 (+ 392us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:13.173560 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:13.173689 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:13.174216 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:13.174219 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:13.174233 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:13.174375 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:13.174381 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.175296 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.175301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:13.176586 (+ 1285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:13.176598 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.177692 (+ 1094us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.177696 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:13.177847 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:13.177879 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:09:13.177966 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:09:13.178047 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":23,"thread_start_us":103,"threads_started":1}
I20260811 22:09:13.178284 30101 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:13.173300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:44447 (local address 127.28.47.131:35835)
0811 22:09:13.173523 (+ 223us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:13.173530 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:13.173652 (+ 122us) server_negotiation.cc:407] Connection header received
0811 22:09:13.173849 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:13.173853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:13.173908 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:13.174012 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:13.174518 (+ 506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.175177 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.176744 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.177525 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.177693 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:13.177780 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:13.178004 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 22:09:13.178107 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 22:09:13.179283 30102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:13.173937 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:45425 (local address 127.28.47.129:46345)
0811 22:09:13.174127 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:13.174134 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:13.174197 (+ 63us) server_negotiation.cc:407] Connection header received
0811 22:09:13.174362 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:13.174368 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:13.174434 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:13.174549 (+ 115us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:13.174896 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.175830 (+ 934us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.177523 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.178410 (+ 887us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.178554 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:13.178610 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:13.179037 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 22:09:13.179131 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:09:13.179445 30032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d971a64a1784c0a84ed725a6babab03" candidate_uuid: "dad1febb1e324cf49491f20fdfe13af2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "876ab7d7852f4548b3d595e8ec8fb280" is_pre_election: true
I20260811 22:09:13.179836 30115 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:13.173723 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:46345 (local address 127.28.47.130:45425)
0811 22:09:13.174064 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:13.174078 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:13.174229 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:13.174558 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:13.174561 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:13.174583 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:13.174754 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:13.174762 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.176000 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.176015 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:13.177380 (+ 1365us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:13.177391 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.178653 (+ 1262us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.178658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:13.178806 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:13.178836 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:09:13.179516 (+ 680us) client_negotiation.cc:240] Negotiation successful
0811 22:09:13.179605 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":275,"mutex_wait_us":31,"thread_start_us":89,"threads_started":1}
I20260811 22:09:13.180022 30032 raft_consensus.cc:2454] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad1febb1e324cf49491f20fdfe13af2 in term 0.
I20260811 22:09:13.180677 29713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d971a64a1784c0a84ed725a6babab03" candidate_uuid: "dad1febb1e324cf49491f20fdfe13af2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cf37584e09b4a6db362f1eeeb18bfd3" is_pre_election: true
I20260811 22:09:13.180799 29795 leader_election.cc:302] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 876ab7d7852f4548b3d595e8ec8fb280, dad1febb1e324cf49491f20fdfe13af2; no voters:
I20260811 22:09:13.181043 29713 raft_consensus.cc:2454] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dad1febb1e324cf49491f20fdfe13af2 in term 0.
I20260811 22:09:13.181270 30113 raft_consensus.cc:2781] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:13.181385 30113 raft_consensus.cc:482] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:13.181449 30113 raft_consensus.cc:3037] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:13.183022 30113 raft_consensus.cc:504] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad1febb1e324cf49491f20fdfe13af2" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37161 } } peers { permanent_uuid: "876ab7d7852f4548b3d595e8ec8fb280" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35835 } } peers { permanent_uuid: "5cf37584e09b4a6db362f1eeeb18bfd3" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 46345 } }
I20260811 22:09:13.183457 30113 leader_election.cc:288] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [CANDIDATE]: Term 1 election: Requested vote from peers 876ab7d7852f4548b3d595e8ec8fb280 (127.28.47.131:35835), 5cf37584e09b4a6db362f1eeeb18bfd3 (127.28.47.129:46345)
I20260811 22:09:13.183982 30032 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d971a64a1784c0a84ed725a6babab03" candidate_uuid: "dad1febb1e324cf49491f20fdfe13af2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "876ab7d7852f4548b3d595e8ec8fb280"
I20260811 22:09:13.184013 29713 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9d971a64a1784c0a84ed725a6babab03" candidate_uuid: "dad1febb1e324cf49491f20fdfe13af2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5cf37584e09b4a6db362f1eeeb18bfd3"
I20260811 22:09:13.184139 30032 raft_consensus.cc:3037] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:13.184139 29713 raft_consensus.cc:3037] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:13.186307 29713 raft_consensus.cc:2454] T 9d971a64a1784c0a84ed725a6babab03 P 5cf37584e09b4a6db362f1eeeb18bfd3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad1febb1e324cf49491f20fdfe13af2 in term 1.
I20260811 22:09:13.186307 30032 raft_consensus.cc:2454] T 9d971a64a1784c0a84ed725a6babab03 P 876ab7d7852f4548b3d595e8ec8fb280 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dad1febb1e324cf49491f20fdfe13af2 in term 1.
I20260811 22:09:13.186718 29795 leader_election.cc:302] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5cf37584e09b4a6db362f1eeeb18bfd3, dad1febb1e324cf49491f20fdfe13af2; no voters:
I20260811 22:09:13.186905 30113 raft_consensus.cc:2781] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:13.187156 30113 raft_consensus.cc:686] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [term 1 LEADER]: Becoming Leader. State: Replica: dad1febb1e324cf49491f20fdfe13af2, State: Running, Role: LEADER
I20260811 22:09:13.187558 30113 consensus_queue.cc:237] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad1febb1e324cf49491f20fdfe13af2" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37161 } } peers { permanent_uuid: "876ab7d7852f4548b3d595e8ec8fb280" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35835 } } peers { permanent_uuid: "5cf37584e09b4a6db362f1eeeb18bfd3" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 46345 } }
I20260811 22:09:13.191635 29540 catalog_manager.cc:5495] T 9d971a64a1784c0a84ed725a6babab03 P dad1febb1e324cf49491f20fdfe13af2 reported cstate change: term changed from 0 to 1, leader changed from <none> to dad1febb1e324cf49491f20fdfe13af2 (127.28.47.130). New cstate: current_term: 1 leader_uuid: "dad1febb1e324cf49491f20fdfe13af2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "dad1febb1e324cf49491f20fdfe13af2" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37161 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "876ab7d7852f4548b3d595e8ec8fb280" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35835 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5cf37584e09b4a6db362f1eeeb18bfd3" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 46345 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:09:13 dist-test-slave-vxj2 krb5kdc[29476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486151, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.129@KRBTEST.COM
I20260811 22:09:13.233371 30102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:13.218509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58766 (local address 127.28.47.129:46345)
0811 22:09:13.218688 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:13.218695 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:13.219071 (+ 376us) server_negotiation.cc:407] Connection header received
0811 22:09:13.219239 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:13.219244 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:13.219316 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:13.219383 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:13.220525 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.221616 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.223030 (+ 1414us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.223515 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.223683 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:13.227215 (+ 3532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:13.227238 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:13.227269 (+ 31us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:13.227322 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:13.230426 (+ 3104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:13.231060 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:13.231071 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:13.231075 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:13.231347 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:13.231818 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:13.231823 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:13.231825 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:13.232266 (+ 441us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:13.232543 (+ 277us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:13.232865 (+ 322us) server_negotiation.cc:299] Negotiation successful
0811 22:09:13.233043 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 22:09:13 dist-test-slave-vxj2 krb5kdc[29476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486153, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:09:13 dist-test-slave-vxj2 krb5kdc[29476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486153, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.47.129@KRBTEST.COM
I20260811 22:09:13.272259 30102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:13.261185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58776 (local address 127.28.47.129:46345)
0811 22:09:13.261334 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:13.261337 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:13.261675 (+ 338us) server_negotiation.cc:407] Connection header received
0811 22:09:13.261857 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:13.261860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:13.261964 (+ 104us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:13.262217 (+ 253us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:13.263118 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.263793 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.264714 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.264951 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.265069 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:13.268498 (+ 3429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:13.268501 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:13.268504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:13.268541 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:13.270353 (+ 1812us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:13.270868 (+ 515us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:13.270873 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:13.270875 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:13.270970 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:13.271274 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:13.271278 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:13.271280 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:13.271481 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:13.271625 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:13.271941 (+ 316us) server_negotiation.cc:299] Negotiation successful
0811 22:09:13.272076 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
Aug 11 22:09:13 dist-test-slave-vxj2 krb5kdc[29476](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:09:13 dist-test-slave-vxj2 krb5kdc[29476](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486153, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.129@KRBTEST.COM
I20260811 22:09:13.303033 30102 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:13.293263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58790 (local address 127.28.47.129:46345)
0811 22:09:13.293431 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:13.293437 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:13.293720 (+ 283us) server_negotiation.cc:407] Connection header received
0811 22:09:13.293949 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:13.293952 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:13.294001 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:13.294087 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:13.294845 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.295386 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.296345 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:13.296564 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:13.296672 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:13.299610 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:13.299614 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:13.299616 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:13.299649 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:13.301138 (+ 1489us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:13.301689 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:13.301692 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:13.301694 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:13.301791 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:13.302111 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:13.302114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:13.302115 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:13.302398 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:13.302518 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:13.302768 (+ 250us) server_negotiation.cc:299] Negotiation successful
0811 22:09:13.302904 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 22:09:13.304930 29663 tablet_service.cc:2908] Rejecting scan request for tablet 9d971a64a1784c0a84ed725a6babab03: Uninitialized: clean time has not yet been initialized
I20260811 22:09:13.307780 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 29613
I20260811 22:09:13.317461 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 29771
I20260811 22:09:13.327829 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 29931
I20260811 22:09:13.338681 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 29492
2026-08-11T22:09:13Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2864 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:13 dist-test-slave-vxj2 krb5kdc[30134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:13 dist-test-slave-vxj2 krb5kdc[30134](info): set up 2 sockets
Aug 11 22:09:13 dist-test-slave-vxj2 krb5kdc[30134](info): commencing operation
krb5kdc: starting...
W20260811 22:09:14.547843 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.180s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
W20260811 22:09:14.774312 28862 mini_kdc.cc:253] Time spent creating user principal test-user: real 0.151s user 0.000s sys 0.002s
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
W20260811 22:09:14.945401 28862 mini_kdc.cc:253] Time spent creating user principal joe-interloper: real 0.171s user 0.000s sys 0.002s
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/test-user.keytab.
W20260811 22:09:15.623039 28862 mini_kdc.cc:302] Time spent creating keytab for test-user: real 0.678s user 0.000s sys 0.002s
Aug 11 22:09:15 dist-test-slave-vxj2 krb5kdc[30134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
W20260811 22:09:16.337239 28862 mini_kdc.cc:262] Time spent creating service keytab for kudu/127.28.47.190: real 0.686s user 0.004s sys 0.000s
I20260811 22:09:16.337870 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45911
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:45705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:45911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:16.466889 30151 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:16.467315 30151 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:16.467423 30151 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:16.472087 30151 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:16.472232 30151 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:09:16.472262 30151 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:16.472294 30151 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:16.472339 30151 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:16.472385 30151 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:16.478579 30151 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:45705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:45911
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30151
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:16.479936 30151 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:16.481143 30151 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:16.492456 30157 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:16.492465 30158 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:16.492465 30161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:16.492511 30151 server_base.cc:1034] running on GCE node
I20260811 22:09:16.493465 30151 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:16.494838 30151 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:16.496078 30151 hybrid_clock.cc:648] HybridClock initialized: now 1786486156496038 us; error 57 us; skew 500 ppm
Aug 11 22:09:16 dist-test-slave-vxj2 krb5kdc[30134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486156, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:16.499493 30151 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:09:16.500717 30151 webserver.cc:460] Webserver started at http://127.28.47.190:37727/ using document root <none> and password file <none>
I20260811 22:09:16.501449 30151 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:16.501554 30151 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:16.501813 30151 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:16.504236 30151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "a955af32ed7d436da5c62286f0ed5eab"
format_stamp: "Formatted at 2026-08-11 22:09:16 on dist-test-slave-vxj2"
server_key: "391f63010db68470e8478d7f11b392ef"
server_key_iv: "b1099e1f3e64dccf612a45548d26f60b"
server_key_version: "encryptionkey@0"
I20260811 22:09:16.504998 30151 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "a955af32ed7d436da5c62286f0ed5eab"
format_stamp: "Formatted at 2026-08-11 22:09:16 on dist-test-slave-vxj2"
server_key: "391f63010db68470e8478d7f11b392ef"
server_key_iv: "b1099e1f3e64dccf612a45548d26f60b"
server_key_version: "encryptionkey@0"
I20260811 22:09:16.510203 30151 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.001s
I20260811 22:09:16.513526 30168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:16.515168 30151 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:09:16.515341 30151 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "a955af32ed7d436da5c62286f0ed5eab"
format_stamp: "Formatted at 2026-08-11 22:09:16 on dist-test-slave-vxj2"
server_key: "391f63010db68470e8478d7f11b392ef"
server_key_iv: "b1099e1f3e64dccf612a45548d26f60b"
server_key_version: "encryptionkey@0"
I20260811 22:09:16.515470 30151 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:16.543026 30151 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:16.546825 30151 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:16.547004 30151 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:16.557282 30151 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:45911
I20260811 22:09:16.557746 30240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:45911 every 8 connection(s)
I20260811 22:09:16.558959 30151 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:16.562486 30241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:16.567315 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 30151
I20260811 22:09:16.567432 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:16.567721 28862 external_mini_cluster.cc:1428] Setting key 1335492b279cae5ac26da7553b99b8c5
I20260811 22:09:16.569693 30241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab: Bootstrap starting.
I20260811 22:09:16.572748 30241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:16.573798 30241 log.cc:824] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:16.576730 30241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab: No bootstrap required, opened a new log
Aug 11 22:09:16 dist-test-slave-vxj2 krb5kdc[30134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:16.582224 30241 raft_consensus.cc:348] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a955af32ed7d436da5c62286f0ed5eab" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45911 } }
I20260811 22:09:16.582638 30241 raft_consensus.cc:374] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:16.582746 30241 raft_consensus.cc:729] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a955af32ed7d436da5c62286f0ed5eab, State: Initialized, Role: FOLLOWER
I20260811 22:09:16.583364 30241 consensus_queue.cc:260] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a955af32ed7d436da5c62286f0ed5eab" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45911 } }
I20260811 22:09:16.583568 30241 raft_consensus.cc:388] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:16.583647 30241 raft_consensus.cc:482] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:16.583750 30241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:16.585199 30241 raft_consensus.cc:504] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a955af32ed7d436da5c62286f0ed5eab" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45911 } }
I20260811 22:09:16.585624 30241 leader_election.cc:302] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a955af32ed7d436da5c62286f0ed5eab; no voters:
I20260811 22:09:16.585934 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:16.569426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35012 (local address 127.28.47.190:45911)
0811 22:09:16.569925 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:16.569947 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:16.569996 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:09:16.570882 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:16.570908 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:16.571400 (+ 492us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:16.571794 (+ 394us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:16.572571 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:16.574183 (+ 1612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:16.575449 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:16.576181 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:16.576386 (+ 205us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:16.579597 (+ 3211us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:16.579604 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:16.579627 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:16.579666 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:16.581974 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:16.582506 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:16.582514 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:16.582519 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:16.582791 (+ 272us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:16.583212 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:16.583215 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:16.583217 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:16.583857 (+ 640us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:16.584068 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:16.584364 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 22:09:16.584717 (+ 353us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":329,"thread_start_us":130,"threads_started":1}
I20260811 22:09:16.586061 30241 leader_election.cc:288] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:16.586225 30247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:16.586581 30247 raft_consensus.cc:686] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [term 1 LEADER]: Becoming Leader. State: Replica: a955af32ed7d436da5c62286f0ed5eab, State: Running, Role: LEADER
I20260811 22:09:16.586923 30247 consensus_queue.cc:237] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a955af32ed7d436da5c62286f0ed5eab" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45911 } }
I20260811 22:09:16.587519 30241 sys_catalog.cc:564] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:16.588219 30247 sys_catalog.cc:455] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [sys.catalog]: SysCatalogTable state changed. Reason: New leader a955af32ed7d436da5c62286f0ed5eab. Latest consensus state: current_term: 1 leader_uuid: "a955af32ed7d436da5c62286f0ed5eab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a955af32ed7d436da5c62286f0ed5eab" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45911 } } }
I20260811 22:09:16.588487 30247 sys_catalog.cc:458] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:16.588642 30248 sys_catalog.cc:455] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a955af32ed7d436da5c62286f0ed5eab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a955af32ed7d436da5c62286f0ed5eab" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45911 } } }
I20260811 22:09:16.588825 30248 sys_catalog.cc:458] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:16.588984 30256 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:16.592203 30256 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:16.603546 30256 catalog_manager.cc:1347] Generated new cluster ID: 45ad8a1ef05048488fdcced8b350edfd
I20260811 22:09:16.603654 30256 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:16.623859 30256 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:16.624950 30256 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:16.634330 30256 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab: Generated new TSK 0
I20260811 22:09:16.635179 30256 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:09:16.673493 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45911
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:45705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:16.827792 30270 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:16.828088 30270 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:16.828137 30270 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:16.832427 30270 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:16.832499 30270 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:16.832541 30270 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:09:16.832621 30270 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:16.839023 30270 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:45705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.47.190:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30270
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:16.840524 30270 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:16.841785 30270 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:16.853014 30280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:16.852916 30278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:16.852911 30277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:16.853430 30270 server_base.cc:1034] running on GCE node
I20260811 22:09:16.853919 30270 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:16.854758 30270 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:16.855969 30270 hybrid_clock.cc:648] HybridClock initialized: now 1786486156855954 us; error 31 us; skew 500 ppm
Aug 11 22:09:16 dist-test-slave-vxj2 krb5kdc[30134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486156, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:16.859575 30270 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:09:16.860826 30270 webserver.cc:460] Webserver started at http://127.28.47.129:45109/ using document root <none> and password file <none>
I20260811 22:09:16.861603 30270 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:16.861714 30270 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:16.861999 30270 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:16.864423 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "d54a9ba0b1a54c29bbc295fae7463cf5"
format_stamp: "Formatted at 2026-08-11 22:09:16 on dist-test-slave-vxj2"
server_key: "5b2197e6330d6991f231964969567d51"
server_key_iv: "5802b169447f16ead9e4d8a72b9996a0"
server_key_version: "encryptionkey@0"
I20260811 22:09:16.865098 30270 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "d54a9ba0b1a54c29bbc295fae7463cf5"
format_stamp: "Formatted at 2026-08-11 22:09:16 on dist-test-slave-vxj2"
server_key: "5b2197e6330d6991f231964969567d51"
server_key_iv: "5802b169447f16ead9e4d8a72b9996a0"
server_key_version: "encryptionkey@0"
I20260811 22:09:16.869746 30270 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:09:16.873088 30288 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:16.874642 30270 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:16.874836 30270 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "d54a9ba0b1a54c29bbc295fae7463cf5"
format_stamp: "Formatted at 2026-08-11 22:09:16 on dist-test-slave-vxj2"
server_key: "5b2197e6330d6991f231964969567d51"
server_key_iv: "5802b169447f16ead9e4d8a72b9996a0"
server_key_version: "encryptionkey@0"
I20260811 22:09:16.875026 30270 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:16.892786 30270 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:16.896458 30270 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:16.896826 30270 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:16.898885 30270 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:16.899001 30270 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:16.899087 30270 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:16.899134 30270 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:09:16 dist-test-slave-vxj2 krb5kdc[30134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486156, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:16.912498 30270 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:42657
I20260811 22:09:16.912565 30419 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:42657 every 8 connection(s)
I20260811 22:09:16.913972 30270 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:16.916545 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:16.900218 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:48171 (local address 127.28.47.190:45911)
0811 22:09:16.900395 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:16.900401 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:16.901878 (+ 1477us) server_negotiation.cc:407] Connection header received
0811 22:09:16.903425 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:16.903432 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:16.903507 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:16.903635 (+ 128us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:16.905108 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:16.905823 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:16.907121 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:16.907390 (+ 269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:16.907513 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:16.911801 (+ 4288us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:16.911810 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:16.911812 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:16.911844 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:16.913944 (+ 2100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:16.914727 (+ 783us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:16.914731 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:16.914733 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:16.914846 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:16.915282 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:16.915284 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:16.915285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:16.915541 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:16.915628 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:16.916263 (+ 635us) server_negotiation.cc:299] Negotiation successful
0811 22:09:16.916413 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:09:16.917578 30298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:16.900738 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45911 (local address 127.28.47.129:48171)
0811 22:09:16.901648 (+ 910us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:16.901718 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:16.903065 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:16.903853 (+ 788us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:16.903860 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:16.904239 (+ 379us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:16.904921 (+ 682us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:16.904938 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:16.905978 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:16.905981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:16.906948 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:16.906959 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:16.907581 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:16.907585 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:16.907774 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:16.908526 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:16.908545 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:16.911550 (+ 3005us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:16.914146 (+ 2596us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:16.914160 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:16.914178 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:16.914578 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:16.914953 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:16.914956 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:16.914958 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:16.915165 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:16.915680 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:16.915686 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:16.916005 (+ 319us) client_negotiation.cc:769] Sending connection context
0811 22:09:16.916188 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 22:09:16.916496 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":643,"thread_start_us":236,"threads_started":1}
I20260811 22:09:16.919101 30420 heartbeater.cc:347] Connected to a master server at 127.28.47.190:45911
I20260811 22:09:16.919274 30420 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:16.919952 30420 heartbeater.cc:511] Master 127.28.47.190:45911 requested a full tablet report, sending...
I20260811 22:09:16.921994 30187 ts_manager.cc:194] Registered new tserver with Master: d54a9ba0b1a54c29bbc295fae7463cf5 (127.28.47.129:42657)
I20260811 22:09:16.923208 30187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:48171
I20260811 22:09:16.923686 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 30270
I20260811 22:09:16.923815 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:09:16.924134 28862 external_mini_cluster.cc:1428] Setting key 710bbdcc192743bbd81bbc63437c577b
I20260811 22:09:16.932089 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:16.921967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:43063 (local address 127.28.47.190:45911)
0811 22:09:16.924038 (+ 2071us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:16.924043 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:16.924356 (+ 313us) server_negotiation.cc:407] Connection header received
0811 22:09:16.924479 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:16.924481 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:16.924567 (+ 86us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:16.924662 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:16.925291 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:16.926024 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:16.926916 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:16.927150 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:16.927263 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:16.928567 (+ 1304us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:16.928571 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:16.928574 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:16.928608 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:16.930296 (+ 1688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:16.930767 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:16.930771 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:16.930772 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:16.930861 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:16.931165 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:16.931169 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:16.931171 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:16.931386 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:16.931502 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:16.931786 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 22:09:16.931915 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1974}
I20260811 22:09:16.932101 30426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:16.921860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45911 (local address 127.28.47.129:43063)
0811 22:09:16.924243 (+ 2383us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:16.924262 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:16.924383 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:16.924710 (+ 327us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:16.924713 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:16.924973 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:16.925136 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:16.925145 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:16.926172 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:16.926176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:16.926775 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:16.926784 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:16.927274 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:16.927278 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:16.927373 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:16.928013 (+ 640us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:16.928025 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:16.928415 (+ 390us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:16.930426 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:16.930431 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:16.930433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:16.930654 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:16.930961 (+ 307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:16.930965 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:16.930967 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:16.931069 (+ 102us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:16.931492 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:16.931495 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:16.931606 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 22:09:16.931720 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:09:16.931903 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2283,"mutex_wait_us":1,"thread_start_us":125,"threads_started":1}
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:09:16.957657 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45911
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:45705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:17.080273 30429 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:17.080610 30429 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:17.080662 30429 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:17.085449 30429 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:17.085533 30429 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:17.085577 30429 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:09:17.085639 30429 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:17.091812 30429 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:45705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.47.190:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30429
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:17.093040 30429 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:17.094568 30429 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:17.105973 30436 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:17.106220 30429 server_base.cc:1034] running on GCE node
W20260811 22:09:17.105973 30439 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:17.105973 30435 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:17.106927 30429 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:17.107739 30429 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:17.108958 30429 hybrid_clock.cc:648] HybridClock initialized: now 1786486157108942 us; error 31 us; skew 500 ppm
Aug 11 22:09:17 dist-test-slave-vxj2 krb5kdc[30134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486157, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:17.112279 30429 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:09:17.113440 30429 webserver.cc:460] Webserver started at http://127.28.47.130:33073/ using document root <none> and password file <none>
I20260811 22:09:17.114076 30429 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:17.114178 30429 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:17.114441 30429 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:17.116773 30429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "55267f359cd3407ba8e19fd2e891cfba"
format_stamp: "Formatted at 2026-08-11 22:09:17 on dist-test-slave-vxj2"
server_key: "5b4c3117f3cba0b344c1e5133c39ba56"
server_key_iv: "64e7d8e19cc5d1bb63fb47339393b880"
server_key_version: "encryptionkey@0"
I20260811 22:09:17.117437 30429 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "55267f359cd3407ba8e19fd2e891cfba"
format_stamp: "Formatted at 2026-08-11 22:09:17 on dist-test-slave-vxj2"
server_key: "5b4c3117f3cba0b344c1e5133c39ba56"
server_key_iv: "64e7d8e19cc5d1bb63fb47339393b880"
server_key_version: "encryptionkey@0"
I20260811 22:09:17.122124 30429 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:09:17.125125 30448 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:17.126860 30429 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:09:17.127054 30429 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "55267f359cd3407ba8e19fd2e891cfba"
format_stamp: "Formatted at 2026-08-11 22:09:17 on dist-test-slave-vxj2"
server_key: "5b4c3117f3cba0b344c1e5133c39ba56"
server_key_iv: "64e7d8e19cc5d1bb63fb47339393b880"
server_key_version: "encryptionkey@0"
I20260811 22:09:17.127203 30429 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:17.145181 30429 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:17.149047 30429 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:17.149330 30429 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:17.151118 30429 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:17.151208 30429 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:17.151280 30429 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:17.151331 30429 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:09:17 dist-test-slave-vxj2 krb5kdc[30134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486157, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:17.163359 30429 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:41621
I20260811 22:09:17.163470 30591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:41621 every 8 connection(s)
I20260811 22:09:17.164817 30429 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:17.166024 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 30429
I20260811 22:09:17.166147 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:09:17.166486 28862 external_mini_cluster.cc:1428] Setting key 71661b3dd9e18a996eebcf391613907c
I20260811 22:09:17.168118 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.152056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:43425 (local address 127.28.47.190:45911)
0811 22:09:17.152234 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.152240 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.153332 (+ 1092us) server_negotiation.cc:407] Connection header received
0811 22:09:17.154465 (+ 1133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.154469 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.154540 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.154640 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:17.156609 (+ 1969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.157503 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.158955 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.159205 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.159332 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.162484 (+ 3152us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:17.162492 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:17.162498 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:17.162542 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:17.164976 (+ 2434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:17.166037 (+ 1061us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:17.166045 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:17.166050 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:17.166171 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:17.166761 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:17.166764 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:17.166766 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:17.167050 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:17.167172 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.167795 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.167950 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:09:17.169083 30463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.152593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45911 (local address 127.28.47.130:43425)
0811 22:09:17.153099 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:17.153191 (+ 92us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:17.154193 (+ 1002us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:17.154914 (+ 721us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:17.154925 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:17.155518 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:17.156365 (+ 847us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.156391 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.157648 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.157651 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.158809 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.158821 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.159403 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.159407 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.159582 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.160270 (+ 688us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:17.160290 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:17.162234 (+ 1944us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:17.165146 (+ 2912us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:17.165192 (+ 46us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:17.165215 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:17.165638 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:17.166401 (+ 763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:17.166405 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:17.166407 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:17.166620 (+ 213us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:17.167201 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:17.167208 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:17.167514 (+ 306us) client_negotiation.cc:769] Sending connection context
0811 22:09:17.167722 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 22:09:17.168148 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":148,"threads_started":1}
I20260811 22:09:17.171105 30592 heartbeater.cc:347] Connected to a master server at 127.28.47.190:45911
I20260811 22:09:17.171293 30592 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:17.172003 30592 heartbeater.cc:511] Master 127.28.47.190:45911 requested a full tablet report, sending...
I20260811 22:09:17.174717 30187 ts_manager.cc:194] Registered new tserver with Master: 55267f359cd3407ba8e19fd2e891cfba (127.28.47.130:41621)
I20260811 22:09:17.175648 30187 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:43425
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:09:17.182372 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.173751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:53009 (local address 127.28.47.190:45911)
0811 22:09:17.173927 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.173934 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.174247 (+ 313us) server_negotiation.cc:407] Connection header received
0811 22:09:17.174438 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.174441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.174496 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.174633 (+ 137us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:17.175151 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.175716 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.176834 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.177080 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.177228 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.178444 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:17.178448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:17.178451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:17.178483 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:17.180369 (+ 1886us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:17.180855 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:17.180858 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:17.180860 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:17.180955 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:17.181292 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:17.181296 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:17.181297 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:17.181558 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:17.181733 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.182065 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.182190 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:09:17.182380 30602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.173653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45911 (local address 127.28.47.130:53009)
0811 22:09:17.174148 (+ 495us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:17.174167 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:17.174333 (+ 166us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:17.174647 (+ 314us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:17.174650 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:17.174905 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:17.175019 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.175025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.175864 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.175867 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.176655 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.176664 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.177223 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.177226 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.177349 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.177927 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:17.177940 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:17.178296 (+ 356us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:17.180496 (+ 2200us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:17.180499 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:17.180501 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:17.180729 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:17.181058 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:17.181061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:17.181063 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:17.181203 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:17.181719 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:17.181722 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:17.181845 (+ 123us) client_negotiation.cc:769] Sending connection context
0811 22:09:17.181989 (+ 144us) client_negotiation.cc:240] Negotiation successful
0811 22:09:17.182164 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":378,"mutex_wait_us":1,"thread_start_us":283,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:09:17.201556 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45911
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:45705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:17.328616 30604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:17.328926 30604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:17.328977 30604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:17.333423 30604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:17.333510 30604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:17.333554 30604 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:09:17.333626 30604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:17.339803 30604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:45705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.47.190:45911
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:17.341092 30604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:17.342348 30604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:17.354059 30613 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:17.353987 30610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:17.354048 30611 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:17.354280 30604 server_base.cc:1034] running on GCE node
I20260811 22:09:17.354885 30604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:17.355650 30604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:17.356921 30604 hybrid_clock.cc:648] HybridClock initialized: now 1786486157356884 us; error 49 us; skew 500 ppm
Aug 11 22:09:17 dist-test-slave-vxj2 krb5kdc[30134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486157, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:17.361233 30604 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:09:17.362326 30604 webserver.cc:460] Webserver started at http://127.28.47.131:33827/ using document root <none> and password file <none>
I20260811 22:09:17.362978 30604 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:17.363044 30604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:17.363291 30604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:17.365685 30604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "fc71d4283da649d7bbad4dd854836efe"
format_stamp: "Formatted at 2026-08-11 22:09:17 on dist-test-slave-vxj2"
server_key: "30e23f5d82681ea543f9e97598baf5eb"
server_key_iv: "4c265ec91db2c1733ed08a5b96e5e6ed"
server_key_version: "encryptionkey@0"
I20260811 22:09:17.366317 30604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "fc71d4283da649d7bbad4dd854836efe"
format_stamp: "Formatted at 2026-08-11 22:09:17 on dist-test-slave-vxj2"
server_key: "30e23f5d82681ea543f9e97598baf5eb"
server_key_iv: "4c265ec91db2c1733ed08a5b96e5e6ed"
server_key_version: "encryptionkey@0"
I20260811 22:09:17.370723 30604 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:09:17.373906 30620 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:17.375452 30604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:09:17.375627 30604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "fc71d4283da649d7bbad4dd854836efe"
format_stamp: "Formatted at 2026-08-11 22:09:17 on dist-test-slave-vxj2"
server_key: "30e23f5d82681ea543f9e97598baf5eb"
server_key_iv: "4c265ec91db2c1733ed08a5b96e5e6ed"
server_key_version: "encryptionkey@0"
I20260811 22:09:17.375765 30604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:17.389493 30604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:17.393437 30604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:17.393677 30604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:17.395342 30604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:17.395398 30604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:17.395442 30604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:17.395503 30604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:09:17 dist-test-slave-vxj2 krb5kdc[30134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486157, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:17.407833 30604 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:45357
I20260811 22:09:17.407866 30761 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:45357 every 8 connection(s)
I20260811 22:09:17.408875 30604 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:17.412024 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 30604
I20260811 22:09:17.412194 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:09:17.412276 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.396455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:50481 (local address 127.28.47.190:45911)
0811 22:09:17.396602 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.396606 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.397881 (+ 1275us) server_negotiation.cc:407] Connection header received
0811 22:09:17.399502 (+ 1621us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.399508 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.399580 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.399687 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:17.401602 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.402519 (+ 917us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.403997 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.404399 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.404535 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.407988 (+ 3453us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:17.407992 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:17.407995 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:17.408035 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:17.409827 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:17.410527 (+ 700us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:17.410532 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:17.410534 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:17.410635 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:17.411020 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:17.411024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:17.411025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:17.411255 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:17.411370 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.411922 (+ 552us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.412086 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:09:17.412554 28862 external_mini_cluster.cc:1428] Setting key 1ac81577a842348f69d3c35fb290dfc1
I20260811 22:09:17.413326 30629 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.397064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45911 (local address 127.28.47.131:50481)
0811 22:09:17.397709 (+ 645us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:17.397758 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:17.399165 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:17.399931 (+ 766us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:17.399940 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:17.400521 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:17.401392 (+ 871us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.401418 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.402702 (+ 1284us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.402705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.403868 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.403880 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.404609 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.404613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.404822 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.405566 (+ 744us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:17.405593 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:17.407778 (+ 2185us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:17.409976 (+ 2198us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:17.409983 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:17.409999 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:17.410399 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:17.410737 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:17.410747 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:17.410749 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:17.410907 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:17.411367 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:17.411375 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:17.411656 (+ 281us) client_negotiation.cc:769] Sending connection context
0811 22:09:17.411859 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 22:09:17.412226 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":424,"thread_start_us":170,"threads_started":1}
I20260811 22:09:17.414582 30762 heartbeater.cc:347] Connected to a master server at 127.28.47.190:45911
I20260811 22:09:17.414763 30762 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:17.415408 30762 heartbeater.cc:511] Master 127.28.47.190:45911 requested a full tablet report, sending...
I20260811 22:09:17.416515 30186 ts_manager.cc:194] Registered new tserver with Master: fc71d4283da649d7bbad4dd854836efe (127.28.47.131:45357)
I20260811 22:09:17.417366 30186 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:50481
I20260811 22:09:17.425758 30770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.417057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45911 (local address 127.28.47.131:56675)
0811 22:09:17.417463 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:17.417476 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:17.417597 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:17.417894 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:17.417896 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:17.418168 (+ 272us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:17.418432 (+ 264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.418442 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.419676 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.419679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.420496 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.420507 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.420961 (+ 454us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.420964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.421094 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.421637 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:17.421650 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:17.422019 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:17.423880 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:17.423883 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:17.423884 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:17.424098 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:17.424386 (+ 288us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:17.424389 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:17.424391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:17.424500 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:17.425089 (+ 589us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:17.425092 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:17.425236 (+ 144us) client_negotiation.cc:769] Sending connection context
0811 22:09:17.425367 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 22:09:17.425528 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":156,"threads_started":1}
I20260811 22:09:17.425968 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.417202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:56675 (local address 127.28.47.190:45911)
0811 22:09:17.417337 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.417342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.417554 (+ 212us) server_negotiation.cc:407] Connection header received
0811 22:09:17.417710 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.417714 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.417770 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.417854 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:17.418589 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.419403 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.420610 (+ 1207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.420825 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.420960 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.422161 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:17.422164 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:17.422167 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:17.422192 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:17.423776 (+ 1584us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:17.424185 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:17.424188 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:17.424189 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:17.424270 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:17.424648 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:17.424651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:17.424652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:17.424909 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:17.425001 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.425516 (+ 515us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.425673 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:09:17.427281 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:09:17 dist-test-slave-vxj2 krb5kdc[30134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486157, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:09:17 dist-test-slave-vxj2 krb5kdc[30134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486157, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:17.457077 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.447690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35026 (local address 127.28.47.190:45911)
0811 22:09:17.447828 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.447832 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.447845 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:09:17.447891 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.447893 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.447935 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.448011 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:17.448864 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.449496 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.450413 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.450639 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.450724 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.453517 (+ 2793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:17.453520 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:17.453522 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:17.453553 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:17.455131 (+ 1578us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:17.455624 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:17.455627 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:17.455628 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:17.455721 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:17.456053 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:17.456055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:17.456056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:17.456268 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:17.456383 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.456677 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.456880 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:09:17.460681 30186 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:09:17.463922 30186 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:17.480134 30783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.474126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42126 (local address 127.28.47.130:41621)
0811 22:09:17.474799 (+ 673us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.474810 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.474833 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:09:17.474900 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.474904 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.475064 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.475252 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:17.475796 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.476930 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.478537 (+ 1607us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.479314 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.479437 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.479621 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.479849 (+ 228us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.479961 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":561,"thread_start_us":165,"threads_started":1}
I20260811 22:09:17.480120 30782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.473987 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:41621 (local address 127.0.0.1:42126)
0811 22:09:17.474430 (+ 443us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:17.474489 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:17.474635 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:17.475459 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:17.475466 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:17.475489 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:17.475648 (+ 159us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.475657 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.477060 (+ 1403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.477064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.478415 (+ 1351us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.478425 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.479448 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.479451 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.479625 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.479682 (+ 57us) client_negotiation.cc:769] Sending connection context
0811 22:09:17.479807 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:09:17.479950 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"mutex_wait_us":3,"thread_start_us":133,"threads_started":1}
I20260811 22:09:17.481243 30786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.475348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:42657 (local address 127.0.0.1:39766)
0811 22:09:17.475737 (+ 389us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:17.475751 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:17.475830 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:17.476515 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:17.476519 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:17.476542 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:17.476649 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.476656 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.478253 (+ 1597us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.478257 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.479640 (+ 1383us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.479649 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.480765 (+ 1116us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.480770 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.480889 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.480920 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:09:17.480973 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 22:09:17.481049 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"mutex_wait_us":41,"thread_start_us":92,"threads_started":1}
I20260811 22:09:17.481503 30784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.474945 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:45357 (local address 127.0.0.1:57026)
0811 22:09:17.475249 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:17.475263 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:17.475346 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:17.476137 (+ 791us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:17.476141 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:17.476162 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:17.476279 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.476287 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.477831 (+ 1544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.477835 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.479143 (+ 1308us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.479155 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.481014 (+ 1859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.481018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.481139 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.481197 (+ 58us) client_negotiation.cc:769] Sending connection context
0811 22:09:17.481288 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:09:17.481366 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":234,"mutex_wait_us":3,"thread_start_us":131,"threads_started":1}
I20260811 22:09:17.481997 30785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.475132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57026 (local address 127.28.47.131:45357)
0811 22:09:17.475622 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.475636 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.475663 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:09:17.475760 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.475769 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.475986 (+ 217us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.476159 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:17.476413 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.477640 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.479295 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.480890 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.481002 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.481194 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.481706 (+ 512us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.481824 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":133,"threads_started":1}
I20260811 22:09:17.482024 30787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.475669 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39766 (local address 127.28.47.129:42657)
0811 22:09:17.476067 (+ 398us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.476073 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.476096 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:09:17.476165 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.476170 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.476334 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.476506 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:17.476921 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.478116 (+ 1195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.479799 (+ 1683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.480601 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.481490 (+ 889us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.481684 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.481738 (+ 54us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.481860 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":94,"threads_started":1}
I20260811 22:09:17.484302 30512 tablet_service.cc:1467] Processing CreateTablet for tablet 19f231ee5110408ab4282b5aed9f9dac (DEFAULT_TABLE table=test-table [id=734f4792bc3045b4ac30610e6ad789ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:17.484902 30686 tablet_service.cc:1467] Processing CreateTablet for tablet 19f231ee5110408ab4282b5aed9f9dac (DEFAULT_TABLE table=test-table [id=734f4792bc3045b4ac30610e6ad789ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:17.484900 30347 tablet_service.cc:1467] Processing CreateTablet for tablet 19f231ee5110408ab4282b5aed9f9dac (DEFAULT_TABLE table=test-table [id=734f4792bc3045b4ac30610e6ad789ac]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:17.485543 30512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f231ee5110408ab4282b5aed9f9dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:17.486095 30347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f231ee5110408ab4282b5aed9f9dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:17.486104 30686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 19f231ee5110408ab4282b5aed9f9dac. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:17.496229 30788 tablet_bootstrap.cc:492] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5: Bootstrap starting.
I20260811 22:09:17.496220 30789 tablet_bootstrap.cc:492] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe: Bootstrap starting.
I20260811 22:09:17.499177 30789 tablet_bootstrap.cc:654] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:17.499177 30788 tablet_bootstrap.cc:654] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:17.500344 30788 log.cc:824] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:17.500504 30789 log.cc:824] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:17.500882 30790 tablet_bootstrap.cc:492] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba: Bootstrap starting.
I20260811 22:09:17.503118 30788 tablet_bootstrap.cc:492] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5: No bootstrap required, opened a new log
I20260811 22:09:17.503255 30790 tablet_bootstrap.cc:654] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:17.503305 30789 tablet_bootstrap.cc:492] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe: No bootstrap required, opened a new log
I20260811 22:09:17.503400 30788 ts_tablet_manager.cc:1397] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:09:17.503473 30789 ts_tablet_manager.cc:1397] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:09:17.504290 30790 log.cc:824] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:17.506278 30790 tablet_bootstrap.cc:492] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba: No bootstrap required, opened a new log
I20260811 22:09:17.506542 30790 ts_tablet_manager.cc:1397] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:09:17.507934 30789 raft_consensus.cc:348] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } }
I20260811 22:09:17.508260 30789 raft_consensus.cc:374] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:17.508327 30789 raft_consensus.cc:729] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc71d4283da649d7bbad4dd854836efe, State: Initialized, Role: FOLLOWER
I20260811 22:09:17.508291 30788 raft_consensus.cc:348] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } }
I20260811 22:09:17.508571 30788 raft_consensus.cc:374] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:17.508637 30788 raft_consensus.cc:729] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d54a9ba0b1a54c29bbc295fae7463cf5, State: Initialized, Role: FOLLOWER
I20260811 22:09:17.508900 30789 consensus_queue.cc:260] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } }
I20260811 22:09:17.509057 30788 consensus_queue.cc:260] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } }
I20260811 22:09:17.509730 30762 heartbeater.cc:503] Master 127.28.47.190:45911 was elected leader, sending a full tablet report...
I20260811 22:09:17.510128 30788 ts_tablet_manager.cc:1428] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5: Time spent starting tablet: real 0.007s user 0.004s sys 0.004s
I20260811 22:09:17.510164 30790 raft_consensus.cc:348] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } }
I20260811 22:09:17.510476 30790 raft_consensus.cc:374] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:17.510521 30420 heartbeater.cc:503] Master 127.28.47.190:45911 was elected leader, sending a full tablet report...
I20260811 22:09:17.510535 30790 raft_consensus.cc:729] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55267f359cd3407ba8e19fd2e891cfba, State: Initialized, Role: FOLLOWER
I20260811 22:09:17.510978 30790 consensus_queue.cc:260] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } }
I20260811 22:09:17.511570 30789 ts_tablet_manager.cc:1428] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260811 22:09:17.512158 30592 heartbeater.cc:503] Master 127.28.47.190:45911 was elected leader, sending a full tablet report...
I20260811 22:09:17.512491 30790 ts_tablet_manager.cc:1428] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:09:17.658895 30798 raft_consensus.cc:482] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:17.659137 30798 raft_consensus.cc:504] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } }
W20260811 22:09:17.660089 30764 tablet.cc:2369] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:09:17.660444 30798 leader_election.cc:288] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 55267f359cd3407ba8e19fd2e891cfba (127.28.47.130:41621), fc71d4283da649d7bbad4dd854836efe (127.28.47.131:45357)
W20260811 22:09:17.665649 30421 tablet.cc:2369] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:09:17.666229 30785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.661365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:52501 (local address 127.28.47.131:45357)
0811 22:09:17.661509 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.661512 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.661888 (+ 376us) server_negotiation.cc:407] Connection header received
0811 22:09:17.661992 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.661994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.662062 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.662147 (+ 85us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:17.662499 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.663190 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.664624 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.665330 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.665467 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.665526 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.666010 (+ 484us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.666092 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:09:17.666255 30800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.660686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:41621 (local address 127.28.47.129:33843)
0811 22:09:17.661071 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:17.661109 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:17.661328 (+ 219us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:17.661776 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:17.661779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:17.661805 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:17.661973 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.661983 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.663065 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.663068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.664438 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.664451 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.665613 (+ 1162us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.665616 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.665815 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.665866 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 22:09:17.665992 (+ 126us) client_negotiation.cc:240] Negotiation successful
0811 22:09:17.666085 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"mutex_wait_us":28,"thread_start_us":131,"threads_started":1}
I20260811 22:09:17.666275 30783 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.660905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:33843 (local address 127.28.47.130:41621)
0811 22:09:17.661082 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.661089 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.661272 (+ 183us) server_negotiation.cc:407] Connection header received
0811 22:09:17.661468 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.661472 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.661631 (+ 159us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.661755 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:17.662098 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.662930 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.664624 (+ 1694us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.665307 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.665505 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.665563 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.666041 (+ 478us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.666142 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:09:17.666667 30801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.661278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:45357 (local address 127.28.47.129:52501)
0811 22:09:17.661674 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:17.661786 (+ 112us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:17.661889 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:17.662199 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:17.662201 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:17.662220 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:17.662371 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.662382 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.663310 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.663313 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.664473 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.664483 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.665603 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.665607 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.665821 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.665866 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 22:09:17.666404 (+ 538us) client_negotiation.cc:240] Negotiation successful
0811 22:09:17.666510 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":82,"thread_start_us":133,"threads_started":1}
W20260811 22:09:17.666682 30593 tablet.cc:2369] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:09:17.667481 30527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f231ee5110408ab4282b5aed9f9dac" candidate_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55267f359cd3407ba8e19fd2e891cfba" is_pre_election: true
I20260811 22:09:17.667697 30709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f231ee5110408ab4282b5aed9f9dac" candidate_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71d4283da649d7bbad4dd854836efe" is_pre_election: true
I20260811 22:09:17.667986 30527 raft_consensus.cc:2454] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d54a9ba0b1a54c29bbc295fae7463cf5 in term 0.
I20260811 22:09:17.668071 30709 raft_consensus.cc:2454] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d54a9ba0b1a54c29bbc295fae7463cf5 in term 0.
I20260811 22:09:17.668635 30291 leader_election.cc:302] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55267f359cd3407ba8e19fd2e891cfba, d54a9ba0b1a54c29bbc295fae7463cf5; no voters:
I20260811 22:09:17.668960 30798 raft_consensus.cc:2781] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:17.669068 30798 raft_consensus.cc:482] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:17.669116 30798 raft_consensus.cc:3037] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:17.670329 30798 raft_consensus.cc:504] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } }
I20260811 22:09:17.670807 30798 leader_election.cc:288] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [CANDIDATE]: Term 1 election: Requested vote from peers 55267f359cd3407ba8e19fd2e891cfba (127.28.47.130:41621), fc71d4283da649d7bbad4dd854836efe (127.28.47.131:45357)
I20260811 22:09:17.671332 30527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f231ee5110408ab4282b5aed9f9dac" candidate_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55267f359cd3407ba8e19fd2e891cfba"
I20260811 22:09:17.671392 30709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "19f231ee5110408ab4282b5aed9f9dac" candidate_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71d4283da649d7bbad4dd854836efe"
I20260811 22:09:17.671526 30709 raft_consensus.cc:3037] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:17.671525 30527 raft_consensus.cc:3037] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:17.673357 30527 raft_consensus.cc:2454] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d54a9ba0b1a54c29bbc295fae7463cf5 in term 1.
I20260811 22:09:17.674009 30291 leader_election.cc:302] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55267f359cd3407ba8e19fd2e891cfba, d54a9ba0b1a54c29bbc295fae7463cf5; no voters:
I20260811 22:09:17.674152 30709 raft_consensus.cc:2454] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d54a9ba0b1a54c29bbc295fae7463cf5 in term 1.
I20260811 22:09:17.674408 30798 raft_consensus.cc:2781] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:17.674748 30798 raft_consensus.cc:686] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [term 1 LEADER]: Becoming Leader. State: Replica: d54a9ba0b1a54c29bbc295fae7463cf5, State: Running, Role: LEADER
I20260811 22:09:17.675271 30798 consensus_queue.cc:237] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } }
I20260811 22:09:17.679580 30183 catalog_manager.cc:5495] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 reported cstate change: term changed from 0 to 1, leader changed from <none> to d54a9ba0b1a54c29bbc295fae7463cf5 (127.28.47.129). New cstate: current_term: 1 leader_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } health_report { overall_health: HEALTHY } } }
I20260811 22:09:17.733309 30787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.729060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39768 (local address 127.28.47.129:42657)
0811 22:09:17.729224 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.729229 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.729248 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:09:17.729468 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.729471 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.729535 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.729645 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:17.729951 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.730507 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.731559 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.731793 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.731931 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.732277 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:17.732817 (+ 540us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:17.732957 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.733040 (+ 83us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.733107 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:09:17.741634 30527 raft_consensus.cc:1261] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba [term 1 FOLLOWER]: Refusing update from remote peer d54a9ba0b1a54c29bbc295fae7463cf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:17.742514 30798 consensus_queue.cc:1035] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:17.743837 30709 raft_consensus.cc:1261] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe [term 1 FOLLOWER]: Refusing update from remote peer d54a9ba0b1a54c29bbc295fae7463cf5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:17.744747 30798 consensus_queue.cc:1035] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:17.752213 30805 mvcc.cc:205] Tried to move back new op lower bound from 7317447302099005440 to 7317447301841055744. Current Snapshot: MvccSnapshot[applied={T|T < 7317447302099005440}]
I20260811 22:09:17.753110 30809 mvcc.cc:205] Tried to move back new op lower bound from 7317447302099005440 to 7317447301841055744. Current Snapshot: MvccSnapshot[applied={T|T < 7317447302099005440}]
I20260811 22:09:17.753963 30807 mvcc.cc:205] Tried to move back new op lower bound from 7317447302099005440 to 7317447301841055744. Current Snapshot: MvccSnapshot[applied={T|T < 7317447302099005440}]
I20260811 22:09:17.771044 30183 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:17.771359 30183 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:17.774981 30183 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab: Sending DeleteTablet for 3 replicas of tablet 19f231ee5110408ab4282b5aed9f9dac
I20260811 22:09:17.776197 30512 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f231ee5110408ab4282b5aed9f9dac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:17 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:42126
I20260811 22:09:17.776221 30347 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f231ee5110408ab4282b5aed9f9dac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:17 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:39766
I20260811 22:09:17.777074 30686 tablet_service.cc:1514] Processing DeleteTablet for tablet 19f231ee5110408ab4282b5aed9f9dac with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:17 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:57026
I20260811 22:09:17.777338 30817 tablet_replica.cc:330] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5: stopping tablet replica
I20260811 22:09:17.777360 30821 tablet_replica.cc:330] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba: stopping tablet replica
I20260811 22:09:17.777805 30817 raft_consensus.cc:2229] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:09:17.777797 30821 raft_consensus.cc:2229] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:17.778239 30821 raft_consensus.cc:2258] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:17.778451 30817 raft_consensus.cc:2258] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:17.778914 30183 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35026:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:09:17.779507 30183 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:17.780447 30821 ts_tablet_manager.cc:1905] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:17.781241 30822 tablet_replica.cc:330] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe: stopping tablet replica
I20260811 22:09:17.781265 30817 ts_tablet_manager.cc:1905] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:17.781654 30822 raft_consensus.cc:2229] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:17.782051 30822 raft_consensus.cc:2258] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:17.785704 30821 ts_tablet_manager.cc:1918] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:17.785857 30821 log.cc:1196] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/19f231ee5110408ab4282b5aed9f9dac
I20260811 22:09:17.786267 30821 ts_tablet_manager.cc:1939] T 19f231ee5110408ab4282b5aed9f9dac P 55267f359cd3407ba8e19fd2e891cfba: Deleting consensus metadata
I20260811 22:09:17.786590 30347 tablet_service.cc:1467] Processing CreateTablet for tablet ce6bcb361ba14e4b86acdb109987a712 (DEFAULT_TABLE table=test-table [id=bda7709e0f304a0e9cc87455e7d7571f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:17.786960 30347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce6bcb361ba14e4b86acdb109987a712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:17.788462 30817 ts_tablet_manager.cc:1918] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:17.788585 30512 tablet_service.cc:1467] Processing CreateTablet for tablet ce6bcb361ba14e4b86acdb109987a712 (DEFAULT_TABLE table=test-table [id=bda7709e0f304a0e9cc87455e7d7571f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:17.788859 30512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce6bcb361ba14e4b86acdb109987a712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:17.788589 30817 log.cc:1196] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/wal/wals/19f231ee5110408ab4282b5aed9f9dac
I20260811 22:09:17.789057 30822 ts_tablet_manager.cc:1905] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:17.789352 30817 ts_tablet_manager.cc:1939] T 19f231ee5110408ab4282b5aed9f9dac P d54a9ba0b1a54c29bbc295fae7463cf5: Deleting consensus metadata
I20260811 22:09:17.790489 30171 catalog_manager.cc:4841] TS d54a9ba0b1a54c29bbc295fae7463cf5 (127.28.47.129:42657): tablet 19f231ee5110408ab4282b5aed9f9dac (table test-table [id=734f4792bc3045b4ac30610e6ad789ac]) successfully deleted
I20260811 22:09:17.791060 30686 tablet_service.cc:1467] Processing CreateTablet for tablet ce6bcb361ba14e4b86acdb109987a712 (DEFAULT_TABLE table=test-table [id=bda7709e0f304a0e9cc87455e7d7571f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:17.791378 30686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ce6bcb361ba14e4b86acdb109987a712. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:17.791846 30790 tablet_bootstrap.cc:492] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba: Bootstrap starting.
I20260811 22:09:17.792876 30822 ts_tablet_manager.cc:1918] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:17.793015 30822 log.cc:1196] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/19f231ee5110408ab4282b5aed9f9dac
I20260811 22:09:17.793578 30790 tablet_bootstrap.cc:654] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:17.794010 30788 tablet_bootstrap.cc:492] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5: Bootstrap starting.
I20260811 22:09:17.794199 30789 tablet_bootstrap.cc:492] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe: Bootstrap starting.
I20260811 22:09:17.795892 30788 tablet_bootstrap.cc:654] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:17.796137 30790 tablet_bootstrap.cc:492] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba: No bootstrap required, opened a new log
I20260811 22:09:17.796216 30790 ts_tablet_manager.cc:1397] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 22:09:17.796351 30789 tablet_bootstrap.cc:654] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:17.796969 30790 raft_consensus.cc:348] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.797204 30790 raft_consensus.cc:374] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:17.797247 30790 raft_consensus.cc:729] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55267f359cd3407ba8e19fd2e891cfba, State: Initialized, Role: FOLLOWER
I20260811 22:09:17.797410 30790 consensus_queue.cc:260] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.797689 30790 ts_tablet_manager.cc:1428] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:09:17.798328 30171 catalog_manager.cc:4841] TS 55267f359cd3407ba8e19fd2e891cfba (127.28.47.130:41621): tablet 19f231ee5110408ab4282b5aed9f9dac (table test-table [id=734f4792bc3045b4ac30610e6ad789ac]) successfully deleted
I20260811 22:09:17.799588 30789 tablet_bootstrap.cc:492] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe: No bootstrap required, opened a new log
I20260811 22:09:17.799752 30789 ts_tablet_manager.cc:1397] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:09:17.800035 30822 ts_tablet_manager.cc:1939] T 19f231ee5110408ab4282b5aed9f9dac P fc71d4283da649d7bbad4dd854836efe: Deleting consensus metadata
I20260811 22:09:17.800297 30789 raft_consensus.cc:348] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.800676 30789 raft_consensus.cc:374] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:17.800725 30789 raft_consensus.cc:729] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc71d4283da649d7bbad4dd854836efe, State: Initialized, Role: FOLLOWER
I20260811 22:09:17.800900 30789 consensus_queue.cc:260] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.801301 30799 raft_consensus.cc:482] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:17.801432 30799 raft_consensus.cc:504] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.802085 30171 catalog_manager.cc:4841] TS fc71d4283da649d7bbad4dd854836efe (127.28.47.131:45357): tablet 19f231ee5110408ab4282b5aed9f9dac (table test-table [id=734f4792bc3045b4ac30610e6ad789ac]) successfully deleted
I20260811 22:09:17.802315 30789 ts_tablet_manager.cc:1428] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 22:09:17.802793 30799 leader_election.cc:288] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d54a9ba0b1a54c29bbc295fae7463cf5 (127.28.47.129:42657), fc71d4283da649d7bbad4dd854836efe (127.28.47.131:45357)
I20260811 22:09:17.804790 30788 tablet_bootstrap.cc:492] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5: No bootstrap required, opened a new log
I20260811 22:09:17.804940 30788 ts_tablet_manager.cc:1397] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5: Time spent bootstrapping tablet: real 0.011s user 0.003s sys 0.000s
I20260811 22:09:17.805807 30788 raft_consensus.cc:348] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.805963 30788 raft_consensus.cc:374] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:17.806012 30788 raft_consensus.cc:729] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d54a9ba0b1a54c29bbc295fae7463cf5, State: Initialized, Role: FOLLOWER
I20260811 22:09:17.806188 30788 consensus_queue.cc:260] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.806538 30788 ts_tablet_manager.cc:1428] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5: Time spent starting tablet: real 0.002s user 0.000s sys 0.001s
I20260811 22:09:17.809401 30787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.803724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:41151 (local address 127.28.47.129:42657)
0811 22:09:17.803876 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.803880 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.804168 (+ 288us) server_negotiation.cc:407] Connection header received
0811 22:09:17.804360 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.804363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.804415 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.804514 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:17.805003 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.805964 (+ 961us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.807951 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.808592 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.808686 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.808756 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.809142 (+ 386us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.809268 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 22:09:17.809922 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.802975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:42657 (local address 127.28.47.130:41151)
0811 22:09:17.803845 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:17.803865 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:17.804217 (+ 352us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:17.804546 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:17.804550 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:17.804574 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:17.804823 (+ 249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.804836 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.806107 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.806110 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.807790 (+ 1680us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.807804 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.808885 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.808889 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.808998 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.809023 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:09:17.809547 (+ 524us) client_negotiation.cc:240] Negotiation successful
0811 22:09:17.809643 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"thread_start_us":70,"threads_started":1}
I20260811 22:09:17.811609 30368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce6bcb361ba14e4b86acdb109987a712" candidate_uuid: "55267f359cd3407ba8e19fd2e891cfba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" is_pre_election: true
I20260811 22:09:17.812022 30368 raft_consensus.cc:2454] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55267f359cd3407ba8e19fd2e891cfba in term 0.
I20260811 22:09:17.812623 30451 leader_election.cc:302] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55267f359cd3407ba8e19fd2e891cfba, d54a9ba0b1a54c29bbc295fae7463cf5; no voters:
I20260811 22:09:17.812971 30799 raft_consensus.cc:2781] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:17.813062 30799 raft_consensus.cc:482] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:17.813119 30799 raft_consensus.cc:3037] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:17.814636 30829 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.803313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:45357 (local address 127.28.47.130:42291)
0811 22:09:17.810251 (+ 6938us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:17.810266 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:17.810414 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:17.810813 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:17.810816 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:17.810833 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:17.810923 (+ 90us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.810930 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.811958 (+ 1028us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.811962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.813163 (+ 1201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.813228 (+ 65us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.814168 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.814172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.814278 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.814308 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:09:17.814396 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 22:09:17.814483 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6861,"thread_start_us":103,"threads_started":1}
I20260811 22:09:17.814720 30785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.803894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:42291 (local address 127.28.47.131:45357)
0811 22:09:17.804034 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.804038 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.810328 (+ 6290us) server_negotiation.cc:407] Connection header received
0811 22:09:17.810560 (+ 232us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.810563 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.810616 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.810664 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:17.811050 (+ 386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.811805 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.813340 (+ 1535us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.813976 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.814124 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.814183 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.814462 (+ 279us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.814575 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:09:17.814666 30799 raft_consensus.cc:504] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.815129 30799 leader_election.cc:288] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [CANDIDATE]: Term 1 election: Requested vote from peers d54a9ba0b1a54c29bbc295fae7463cf5 (127.28.47.129:42657), fc71d4283da649d7bbad4dd854836efe (127.28.47.131:45357)
I20260811 22:09:17.815212 30709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce6bcb361ba14e4b86acdb109987a712" candidate_uuid: "55267f359cd3407ba8e19fd2e891cfba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71d4283da649d7bbad4dd854836efe" is_pre_election: true
I20260811 22:09:17.815366 30709 raft_consensus.cc:2454] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55267f359cd3407ba8e19fd2e891cfba in term 0.
I20260811 22:09:17.815961 30368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce6bcb361ba14e4b86acdb109987a712" candidate_uuid: "55267f359cd3407ba8e19fd2e891cfba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5"
I20260811 22:09:17.816237 30368 raft_consensus.cc:3037] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:17.817344 30709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ce6bcb361ba14e4b86acdb109987a712" candidate_uuid: "55267f359cd3407ba8e19fd2e891cfba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71d4283da649d7bbad4dd854836efe"
I20260811 22:09:17.817471 30709 raft_consensus.cc:3037] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:17.817875 30368 raft_consensus.cc:2454] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55267f359cd3407ba8e19fd2e891cfba in term 1.
I20260811 22:09:17.818272 30451 leader_election.cc:302] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55267f359cd3407ba8e19fd2e891cfba, d54a9ba0b1a54c29bbc295fae7463cf5; no voters:
I20260811 22:09:17.818508 30799 raft_consensus.cc:2781] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:17.818832 30709 raft_consensus.cc:2454] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55267f359cd3407ba8e19fd2e891cfba in term 1.
I20260811 22:09:17.819020 30799 raft_consensus.cc:686] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [term 1 LEADER]: Becoming Leader. State: Replica: 55267f359cd3407ba8e19fd2e891cfba, State: Running, Role: LEADER
I20260811 22:09:17.819386 30799 consensus_queue.cc:237] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.822165 30187 catalog_manager.cc:5495] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba reported cstate change: term changed from 0 to 1, leader changed from <none> to 55267f359cd3407ba8e19fd2e891cfba (127.28.47.130). New cstate: current_term: 1 leader_uuid: "55267f359cd3407ba8e19fd2e891cfba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:09:17 dist-test-slave-vxj2 krb5kdc[30134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486156, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:17.894186 30840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.879356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45911 (local address 127.0.0.1:35038)
0811 22:09:17.879824 (+ 468us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:17.879837 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:17.879994 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:17.880249 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:17.880251 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:17.880810 (+ 559us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:17.880921 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.880929 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.881794 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.881800 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.882349 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:17.882357 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.882815 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.882818 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:17.882903 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.888391 (+ 5488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:17.888413 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:17.890322 (+ 1909us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:17.892365 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:17.892380 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:17.892387 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:17.892639 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:17.892938 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:17.892942 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:17.892944 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:17.893034 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:17.893504 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:17.893512 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:17.893638 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 22:09:17.893758 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 22:09:17.893960 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":136,"threads_started":1}
I20260811 22:09:17.894184 30244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:17.879430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35038 (local address 127.28.47.190:45911)
0811 22:09:17.879588 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:17.879594 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:17.879928 (+ 334us) server_negotiation.cc:407] Connection header received
0811 22:09:17.880093 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:17.880096 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:17.880142 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:17.880229 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:17.881048 (+ 819us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.881678 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.882472 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:17.882697 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:17.882805 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:17.890559 (+ 7754us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:17.890563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:17.890566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:17.890601 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:17.892231 (+ 1630us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:17.892755 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:17.892765 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:17.892767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:17.892843 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:17.893130 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:17.893134 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:17.893135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:17.893393 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:17.893504 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:17.893814 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 22:09:17.894028 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":100,"server-negotiator.queue_time_us":57}
I20260811 22:09:17.899477 30187 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:35038:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:09:17.900408 30187 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:17.909518 30347 tablet_service.cc:1467] Processing CreateTablet for tablet e539996315004f56a9a14334b334a8ee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eb286687ad454b45a7a485c70e1900eb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:09:17.909673 30512 tablet_service.cc:1467] Processing CreateTablet for tablet e539996315004f56a9a14334b334a8ee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eb286687ad454b45a7a485c70e1900eb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:09:17.909655 30686 tablet_service.cc:1467] Processing CreateTablet for tablet e539996315004f56a9a14334b334a8ee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=eb286687ad454b45a7a485c70e1900eb]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:09:17.910115 30347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e539996315004f56a9a14334b334a8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:17.910115 30686 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e539996315004f56a9a14334b334a8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:17.910116 30512 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e539996315004f56a9a14334b334a8ee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:17.914031 30789 tablet_bootstrap.cc:492] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe: Bootstrap starting.
I20260811 22:09:17.915475 30789 tablet_bootstrap.cc:654] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:17.916172 30790 tablet_bootstrap.cc:492] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba: Bootstrap starting.
I20260811 22:09:17.917392 30790 tablet_bootstrap.cc:654] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:17.919087 30790 tablet_bootstrap.cc:492] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba: No bootstrap required, opened a new log
I20260811 22:09:17.919185 30790 ts_tablet_manager.cc:1397] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:17.919582 30788 tablet_bootstrap.cc:492] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5: Bootstrap starting.
I20260811 22:09:17.919750 30790 raft_consensus.cc:348] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.919873 30790 raft_consensus.cc:374] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:17.919934 30790 raft_consensus.cc:729] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55267f359cd3407ba8e19fd2e891cfba, State: Initialized, Role: FOLLOWER
I20260811 22:09:17.920401 30790 consensus_queue.cc:260] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.920897 30799 tablet_replica.cc:439] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } }
I20260811 22:09:17.921109 30799 tablet_replica.cc:442] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:17.921701 30788 tablet_bootstrap.cc:654] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:17.923892 30790 ts_tablet_manager.cc:1428] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba: Time spent starting tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:09:17.926903 30789 tablet_bootstrap.cc:492] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe: No bootstrap required, opened a new log
I20260811 22:09:17.927047 30789 ts_tablet_manager.cc:1397] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe: Time spent bootstrapping tablet: real 0.013s user 0.000s sys 0.003s
I20260811 22:09:17.927790 30789 raft_consensus.cc:348] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.927919 30789 raft_consensus.cc:374] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:17.927949 30789 raft_consensus.cc:729] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc71d4283da649d7bbad4dd854836efe, State: Initialized, Role: FOLLOWER
I20260811 22:09:17.928087 30789 consensus_queue.cc:260] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.928346 30789 ts_tablet_manager.cc:1428] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:09:17.928395 30796 tablet_replica.cc:439] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } }
I20260811 22:09:17.928574 30796 tablet_replica.cc:442] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:17.930208 30788 tablet_bootstrap.cc:492] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5: No bootstrap required, opened a new log
I20260811 22:09:17.930330 30788 ts_tablet_manager.cc:1397] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5: Time spent bootstrapping tablet: real 0.011s user 0.004s sys 0.001s
I20260811 22:09:17.931123 30788 raft_consensus.cc:348] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.931255 30788 raft_consensus.cc:374] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:17.931290 30788 raft_consensus.cc:729] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d54a9ba0b1a54c29bbc295fae7463cf5, State: Initialized, Role: FOLLOWER
I20260811 22:09:17.931440 30788 consensus_queue.cc:260] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:17.931725 30788 ts_tablet_manager.cc:1428] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:09:17.931770 30812 tablet_replica.cc:439] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } }
I20260811 22:09:17.932083 30812 tablet_replica.cc:442] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:18.202332 30799 consensus_queue.cc:1035] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:18.207080 30846 consensus_queue.cc:1035] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [LEADER]: Connected to new peer: Peer: permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:18.290753 30846 raft_consensus.cc:482] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:18.290916 30846 raft_consensus.cc:504] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:18.291332 30846 leader_election.cc:288] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d54a9ba0b1a54c29bbc295fae7463cf5 (127.28.47.129:42657), fc71d4283da649d7bbad4dd854836efe (127.28.47.131:45357)
I20260811 22:09:18.291978 30709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e539996315004f56a9a14334b334a8ee" candidate_uuid: "55267f359cd3407ba8e19fd2e891cfba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71d4283da649d7bbad4dd854836efe" is_pre_election: true
I20260811 22:09:18.291980 30368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e539996315004f56a9a14334b334a8ee" candidate_uuid: "55267f359cd3407ba8e19fd2e891cfba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" is_pre_election: true
I20260811 22:09:18.292155 30709 raft_consensus.cc:2454] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55267f359cd3407ba8e19fd2e891cfba in term 0.
I20260811 22:09:18.292156 30368 raft_consensus.cc:2454] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 55267f359cd3407ba8e19fd2e891cfba in term 0.
I20260811 22:09:18.292546 30451 leader_election.cc:302] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55267f359cd3407ba8e19fd2e891cfba, fc71d4283da649d7bbad4dd854836efe; no voters:
I20260811 22:09:18.292794 30846 raft_consensus.cc:2781] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:18.292860 30846 raft_consensus.cc:482] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:18.292891 30846 raft_consensus.cc:3037] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:18.294232 30846 raft_consensus.cc:504] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:18.294625 30846 leader_election.cc:288] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [CANDIDATE]: Term 1 election: Requested vote from peers d54a9ba0b1a54c29bbc295fae7463cf5 (127.28.47.129:42657), fc71d4283da649d7bbad4dd854836efe (127.28.47.131:45357)
I20260811 22:09:18.295080 30368 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e539996315004f56a9a14334b334a8ee" candidate_uuid: "55267f359cd3407ba8e19fd2e891cfba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5"
I20260811 22:09:18.295190 30709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e539996315004f56a9a14334b334a8ee" candidate_uuid: "55267f359cd3407ba8e19fd2e891cfba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fc71d4283da649d7bbad4dd854836efe"
I20260811 22:09:18.295252 30368 raft_consensus.cc:3037] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:18.295333 30709 raft_consensus.cc:3037] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:18.296300 30709 raft_consensus.cc:2454] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55267f359cd3407ba8e19fd2e891cfba in term 1.
I20260811 22:09:18.296646 30368 raft_consensus.cc:2454] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 55267f359cd3407ba8e19fd2e891cfba in term 1.
I20260811 22:09:18.296669 30451 leader_election.cc:302] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55267f359cd3407ba8e19fd2e891cfba, fc71d4283da649d7bbad4dd854836efe; no voters:
I20260811 22:09:18.296854 30846 raft_consensus.cc:2781] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:18.297123 30846 raft_consensus.cc:686] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [term 1 LEADER]: Becoming Leader. State: Replica: 55267f359cd3407ba8e19fd2e891cfba, State: Running, Role: LEADER
I20260811 22:09:18.297314 30846 consensus_queue.cc:237] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } }
I20260811 22:09:18.298650 30799 tablet_replica.cc:439] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba: TxnStatusTablet state changed. Reason: New leader 55267f359cd3407ba8e19fd2e891cfba. Latest consensus state: current_term: 1 leader_uuid: "55267f359cd3407ba8e19fd2e891cfba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } }
I20260811 22:09:18.298797 30799 tablet_replica.cc:442] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:09:18.299203 30849 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:09:18.299454 30849 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:09:18.299921 30187 catalog_manager.cc:5495] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba reported cstate change: term changed from 0 to 1, leader changed from <none> to 55267f359cd3407ba8e19fd2e891cfba (127.28.47.130). New cstate: current_term: 1 leader_uuid: "55267f359cd3407ba8e19fd2e891cfba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } health_report { overall_health: UNKNOWN } } }
I20260811 22:09:18.395782 30840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:18.391293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:41621 (local address 127.0.0.1:42134)
0811 22:09:18.391462 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:18.391482 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:18.391590 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:18.392087 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:18.392090 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:18.392098 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:09:18.392222 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:18.392228 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.393034 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.393037 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:18.393758 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:18.393772 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.394405 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.394409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:18.394519 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:18.394578 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:09:18.395446 (+ 868us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:09:18.395468 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:09:18.395541 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:09:18.395649 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30}
I20260811 22:09:18.395889 30853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:18.391448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42134 (local address 127.28.47.130:41621)
0811 22:09:18.391797 (+ 349us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:18.391801 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:18.391819 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:09:18.391892 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:18.391893 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:18.391949 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:18.392070 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:18.392340 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.392906 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.393904 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.394258 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.394387 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:18.394755 (+ 368us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:18.395345 (+ 590us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:18.395477 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:18.395552 (+ 75us) server_negotiation.cc:299] Negotiation successful
0811 22:09:18.395616 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":268,"thread_start_us":151,"threads_started":1}
I20260811 22:09:18.399483 30709 raft_consensus.cc:1261] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe [term 1 FOLLOWER]: Refusing update from remote peer 55267f359cd3407ba8e19fd2e891cfba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:18.399499 30368 raft_consensus.cc:1261] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5 [term 1 FOLLOWER]: Refusing update from remote peer 55267f359cd3407ba8e19fd2e891cfba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:18.399991 30846 consensus_queue.cc:1035] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [LEADER]: Connected to new peer: Peer: permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:18.400187 30799 consensus_queue.cc:1035] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba [LEADER]: Connected to new peer: Peer: permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:18.402030 30812 tablet_replica.cc:439] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5: TxnStatusTablet state changed. Reason: New leader 55267f359cd3407ba8e19fd2e891cfba. Latest consensus state: current_term: 1 leader_uuid: "55267f359cd3407ba8e19fd2e891cfba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } }
I20260811 22:09:18.402168 30812 tablet_replica.cc:442] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:18.403900 30799 tablet_replica.cc:439] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55267f359cd3407ba8e19fd2e891cfba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } }
I20260811 22:09:18.404034 30799 tablet_replica.cc:442] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:09:18.404110 30796 tablet_replica.cc:439] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe: TxnStatusTablet state changed. Reason: New leader 55267f359cd3407ba8e19fd2e891cfba. Latest consensus state: current_term: 1 leader_uuid: "55267f359cd3407ba8e19fd2e891cfba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } }
I20260811 22:09:18.404273 30796 tablet_replica.cc:442] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:18.405540 30831 tablet_replica.cc:439] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "55267f359cd3407ba8e19fd2e891cfba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } }
I20260811 22:09:18.405678 30831 tablet_replica.cc:442] T e539996315004f56a9a14334b334a8ee P 55267f359cd3407ba8e19fd2e891cfba: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:09:18.405606 30812 tablet_replica.cc:439] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55267f359cd3407ba8e19fd2e891cfba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } }
I20260811 22:09:18.405719 30812 tablet_replica.cc:442] T e539996315004f56a9a14334b334a8ee P d54a9ba0b1a54c29bbc295fae7463cf5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:18.407871 30796 tablet_replica.cc:439] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "55267f359cd3407ba8e19fd2e891cfba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d54a9ba0b1a54c29bbc295fae7463cf5" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42657 } } peers { permanent_uuid: "55267f359cd3407ba8e19fd2e891cfba" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41621 } } peers { permanent_uuid: "fc71d4283da649d7bbad4dd854836efe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45357 } } }
I20260811 22:09:18.408039 30796 tablet_replica.cc:442] T e539996315004f56a9a14334b334a8ee P fc71d4283da649d7bbad4dd854836efe: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:18.419940 30853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:18.415967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42140 (local address 127.28.47.130:41621)
0811 22:09:18.416096 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:18.416103 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:18.416119 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:09:18.416163 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:18.416166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:18.416219 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:18.416304 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:18.416656 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.417251 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.418354 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.418733 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.418983 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:18.419274 (+ 291us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:18.419410 (+ 136us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:18.419476 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:18.419697 (+ 221us) server_negotiation.cc:299] Negotiation successful
0811 22:09:18.419761 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:09:18.433318 30853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:18.427740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:58771 (local address 127.28.47.130:41621)
0811 22:09:18.427877 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:18.427883 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:18.429520 (+ 1637us) server_negotiation.cc:407] Connection header received
0811 22:09:18.429709 (+ 189us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:18.429711 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:18.429764 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:18.429853 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:18.430161 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.430761 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.431873 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.432227 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.432366 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:18.432714 (+ 348us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:18.432802 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:18.432891 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:18.433001 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 22:09:18.433097 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":37}
I20260811 22:09:18.433331 30865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:18.427733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:41621 (local address 127.28.47.130:58771)
0811 22:09:18.429415 (+ 1682us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:18.429429 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:18.429580 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:18.429873 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:18.429875 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:18.429882 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:09:18.430035 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:18.430042 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.430948 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.430954 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:18.431752 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:18.431762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.432378 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.432382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:18.432476 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:18.432552 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:09:18.432892 (+ 340us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:09:18.432900 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:09:18.432982 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 22:09:18.433119 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1569,"mutex_wait_us":22,"thread_start_us":117,"threads_started":1}
I20260811 22:09:18.498595 30187 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:18.498911 30187 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35026:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:18.501936 30187 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a955af32ed7d436da5c62286f0ed5eab: Sending DeleteTablet for 3 replicas of tablet ce6bcb361ba14e4b86acdb109987a712
I20260811 22:09:18.502877 30347 tablet_service.cc:1514] Processing DeleteTablet for tablet ce6bcb361ba14e4b86acdb109987a712 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:18 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:39766
I20260811 22:09:18.503062 30686 tablet_service.cc:1514] Processing DeleteTablet for tablet ce6bcb361ba14e4b86acdb109987a712 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:18 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:57026
I20260811 22:09:18.503531 30869 tablet_replica.cc:330] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe: stopping tablet replica
I20260811 22:09:18.503599 30867 tablet_replica.cc:330] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5: stopping tablet replica
I20260811 22:09:18.503748 30869 raft_consensus.cc:2229] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:18.503743 30867 raft_consensus.cc:2229] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:18.503906 30867 raft_consensus.cc:2258] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:18.503947 30869 raft_consensus.cc:2258] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:18.504767 30869 ts_tablet_manager.cc:1905] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:18.504768 30867 ts_tablet_manager.cc:1905] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:18.504803 30512 tablet_service.cc:1514] Processing DeleteTablet for tablet ce6bcb361ba14e4b86acdb109987a712 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:18 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:42126
I20260811 22:09:18.505597 30871 tablet_replica.cc:330] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba: stopping tablet replica
I20260811 22:09:18.505908 30871 raft_consensus.cc:2229] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:09:18.506443 30871 raft_consensus.cc:2258] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:18.507200 30871 ts_tablet_manager.cc:1905] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:18.508466 30867 ts_tablet_manager.cc:1918] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:18.508466 30869 ts_tablet_manager.cc:1918] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:18.508564 30867 log.cc:1196] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-0/wal/wals/ce6bcb361ba14e4b86acdb109987a712
I20260811 22:09:18.508695 30869 log.cc:1196] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/ce6bcb361ba14e4b86acdb109987a712
I20260811 22:09:18.508867 30867 ts_tablet_manager.cc:1939] T ce6bcb361ba14e4b86acdb109987a712 P d54a9ba0b1a54c29bbc295fae7463cf5: Deleting consensus metadata
I20260811 22:09:18.509130 30869 ts_tablet_manager.cc:1939] T ce6bcb361ba14e4b86acdb109987a712 P fc71d4283da649d7bbad4dd854836efe: Deleting consensus metadata
I20260811 22:09:18.510113 30171 catalog_manager.cc:4841] TS fc71d4283da649d7bbad4dd854836efe (127.28.47.131:45357): tablet ce6bcb361ba14e4b86acdb109987a712 (table test-table [id=bda7709e0f304a0e9cc87455e7d7571f]) successfully deleted
I20260811 22:09:18.510360 30171 catalog_manager.cc:4841] TS d54a9ba0b1a54c29bbc295fae7463cf5 (127.28.47.129:42657): tablet ce6bcb361ba14e4b86acdb109987a712 (table test-table [id=bda7709e0f304a0e9cc87455e7d7571f]) successfully deleted
I20260811 22:09:18.511157 30871 ts_tablet_manager.cc:1918] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:18.513433 30871 log.cc:1196] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/ce6bcb361ba14e4b86acdb109987a712
I20260811 22:09:18.513800 30871 ts_tablet_manager.cc:1939] T ce6bcb361ba14e4b86acdb109987a712 P 55267f359cd3407ba8e19fd2e891cfba: Deleting consensus metadata
I20260811 22:09:18.514688 30171 catalog_manager.cc:4841] TS 55267f359cd3407ba8e19fd2e891cfba (127.28.47.130:41621): tablet ce6bcb361ba14e4b86acdb109987a712 (table test-table [id=bda7709e0f304a0e9cc87455e7d7571f]) successfully deleted
Aug 11 22:09:18 dist-test-slave-vxj2 krb5kdc[30134](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486157, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.47.129@KRBTEST.COM
I20260811 22:09:18.523427 30872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:18.509530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39776 (local address 127.28.47.129:42657)
0811 22:09:18.510783 (+ 1253us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:18.510790 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:18.510870 (+ 80us) server_negotiation.cc:407] Connection header received
0811 22:09:18.511055 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:18.511058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:18.511115 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:18.511227 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:18.512559 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.513096 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.515141 (+ 2045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.515392 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.515513 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:18.519233 (+ 3720us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:18.519291 (+ 58us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:18.519311 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:18.519354 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:18.521462 (+ 2108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:18.521968 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:18.521975 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:18.521976 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:18.522063 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:18.522365 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:18.522369 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:18.522371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:18.522677 (+ 306us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:18.522934 (+ 257us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:18.523095 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 22:09:18.523238 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1172,"thread_start_us":76,"threads_started":1}
W20260811 22:09:18.524104 30413 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:39776
I20260811 22:09:18.536111 30876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:18.526146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35046 (local address 127.28.47.190:45911)
0811 22:09:18.526427 (+ 281us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:18.526430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:18.526456 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:09:18.526522 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:18.526524 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:18.526565 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:18.526641 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:18.527825 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.528404 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.530259 (+ 1855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:18.530610 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:18.530757 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:18.532464 (+ 1707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:18.532469 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:18.532471 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:18.532504 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:18.534187 (+ 1683us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:18.534661 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:18.534665 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:18.534667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:18.534817 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:18.535154 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:18.535157 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:18.535158 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:18.535374 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:18.535528 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:18.535774 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 22:09:18.535918 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":90,"threads_started":1}
W20260811 22:09:18.536613 30187 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35046
I20260811 22:09:18.540019 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 30270
I20260811 22:09:18.567534 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 30429
I20260811 22:09:18.576825 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 30604
I20260811 22:09:18.587733 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 30151
2026-08-11T22:09:18Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (5247 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:18 dist-test-slave-vxj2 krb5kdc[30880](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:18 dist-test-slave-vxj2 krb5kdc[30880](info): set up 2 sockets
Aug 11 22:09:18 dist-test-slave-vxj2 krb5kdc[30880](info): commencing operation
krb5kdc: starting...
W20260811 22:09:19.766692 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:09:19 dist-test-slave-vxj2 krb5kdc[30880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:09:19.909772 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:44257
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:41763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:44257
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:20.034339 30897 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:20.034678 30897 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:20.034732 30897 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:20.038975 30897 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:20.039047 30897 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:09:20.039064 30897 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:20.039083 30897 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:20.039099 30897 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:20.039115 30897 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:20.045651 30897 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:41763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:44257
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:44257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30897
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:20.047411 30897 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:20.049357 30897 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:20.060839 30903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:20.061184 30897 server_base.cc:1034] running on GCE node
W20260811 22:09:20.061213 30904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:20.060906 30906 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:20.062523 30897 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:20.063938 30897 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:20.065246 30897 hybrid_clock.cc:648] HybridClock initialized: now 1786486160065204 us; error 55 us; skew 500 ppm
Aug 11 22:09:20 dist-test-slave-vxj2 krb5kdc[30880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486160, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:20.068770 30897 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:09:20.069932 30897 webserver.cc:460] Webserver started at http://127.28.47.190:43147/ using document root <none> and password file <none>
I20260811 22:09:20.070585 30897 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:20.070686 30897 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:20.070942 30897 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:20.073665 30897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "c122a810b719438eb6cce2cd66a2cc90"
format_stamp: "Formatted at 2026-08-11 22:09:20 on dist-test-slave-vxj2"
server_key: "6acb32fddbdcee534c97a03f44eefe21"
server_key_iv: "481f5578e7bcb7995c28b89ab5071fbc"
server_key_version: "encryptionkey@0"
I20260811 22:09:20.074496 30897 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "c122a810b719438eb6cce2cd66a2cc90"
format_stamp: "Formatted at 2026-08-11 22:09:20 on dist-test-slave-vxj2"
server_key: "6acb32fddbdcee534c97a03f44eefe21"
server_key_iv: "481f5578e7bcb7995c28b89ab5071fbc"
server_key_version: "encryptionkey@0"
I20260811 22:09:20.079198 30897 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:09:20.082289 30914 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:20.083972 30897 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:20.084177 30897 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "c122a810b719438eb6cce2cd66a2cc90"
format_stamp: "Formatted at 2026-08-11 22:09:20 on dist-test-slave-vxj2"
server_key: "6acb32fddbdcee534c97a03f44eefe21"
server_key_iv: "481f5578e7bcb7995c28b89ab5071fbc"
server_key_version: "encryptionkey@0"
I20260811 22:09:20.084335 30897 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:20.102922 30897 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:20.107209 30897 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:20.107484 30897 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:20.117165 30897 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:44257
I20260811 22:09:20.117347 30989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:44257 every 8 connection(s)
I20260811 22:09:20.118280 30897 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:20.121412 30990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:20.128000 30990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90: Bootstrap starting.
I20260811 22:09:20.128544 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 30897
I20260811 22:09:20.128647 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:20.128964 28862 external_mini_cluster.cc:1428] Setting key 40e118d7f1f6c47966bd8a156ec4d40b
I20260811 22:09:20.131124 30990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:20.131954 30990 log.cc:824] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:20.134423 30990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90: No bootstrap required, opened a new log
Aug 11 22:09:20 dist-test-slave-vxj2 krb5kdc[30880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:20.139675 30990 raft_consensus.cc:348] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c122a810b719438eb6cce2cd66a2cc90" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 44257 } }
I20260811 22:09:20.140002 30990 raft_consensus.cc:374] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:20.140089 30990 raft_consensus.cc:729] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c122a810b719438eb6cce2cd66a2cc90, State: Initialized, Role: FOLLOWER
I20260811 22:09:20.140690 30990 consensus_queue.cc:260] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c122a810b719438eb6cce2cd66a2cc90" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 44257 } }
I20260811 22:09:20.140924 30990 raft_consensus.cc:388] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:20.141026 30990 raft_consensus.cc:482] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:20.141186 30990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:20.142665 30990 raft_consensus.cc:504] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c122a810b719438eb6cce2cd66a2cc90" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 44257 } }
I20260811 22:09:20.143110 30990 leader_election.cc:302] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c122a810b719438eb6cce2cd66a2cc90; no voters:
I20260811 22:09:20.143610 30990 leader_election.cc:288] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:20.143698 30996 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:20.143935 30996 raft_consensus.cc:686] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [term 1 LEADER]: Becoming Leader. State: Replica: c122a810b719438eb6cce2cd66a2cc90, State: Running, Role: LEADER
I20260811 22:09:20.144299 30996 consensus_queue.cc:237] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c122a810b719438eb6cce2cd66a2cc90" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 44257 } }
I20260811 22:09:20.145195 30990 sys_catalog.cc:564] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:20.146166 30997 sys_catalog.cc:455] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c122a810b719438eb6cce2cd66a2cc90. Latest consensus state: current_term: 1 leader_uuid: "c122a810b719438eb6cce2cd66a2cc90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c122a810b719438eb6cce2cd66a2cc90" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 44257 } } }
I20260811 22:09:20.146356 30997 sys_catalog.cc:458] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:20.146347 30998 sys_catalog.cc:455] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c122a810b719438eb6cce2cd66a2cc90" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c122a810b719438eb6cce2cd66a2cc90" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 44257 } } }
I20260811 22:09:20.146502 30998 sys_catalog.cc:458] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:20.147044 31006 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:20.147445 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:20.130568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33504 (local address 127.28.47.190:44257)
0811 22:09:20.131059 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:20.131075 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:20.131115 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:09:20.131873 (+ 758us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:20.131892 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:20.132305 (+ 413us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:20.132710 (+ 405us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:20.133647 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.134839 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.136030 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.136620 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.136793 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:20.139973 (+ 3180us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:20.139980 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:20.140016 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:20.140057 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:20.142569 (+ 2512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.143149 (+ 580us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.143156 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.143163 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.143329 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.143640 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.143644 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.143646 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.144338 (+ 692us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:20.145245 (+ 907us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:20.145425 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 22:09:20.145792 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":178,"threads_started":1}
I20260811 22:09:20.150565 31006 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:20.156886 31006 catalog_manager.cc:1347] Generated new cluster ID: 5b92d41fc7a84b58a7d78083a1a7aa60
I20260811 22:09:20.157007 31006 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:20.199067 31006 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:20.200229 31006 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:20.206701 31006 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90: Generated new TSK 0
I20260811 22:09:20.207625 31006 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:09:20.249467 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:44257
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:41763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:20.376185 31019 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:20.376591 31019 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:20.376672 31019 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:20.381274 31019 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:20.381379 31019 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:20.381428 31019 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:09:20.381446 31019 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:09:20.381528 31019 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:20.387796 31019 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:41763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.47.190:44257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31019
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:20.389256 31019 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:20.390501 31019 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:20.401569 31028 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:20.401551 31025 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:20.401551 31026 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:20.401801 31019 server_base.cc:1034] running on GCE node
I20260811 22:09:20.402486 31019 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:20.403365 31019 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:20.404614 31019 hybrid_clock.cc:648] HybridClock initialized: now 1786486160404590 us; error 39 us; skew 500 ppm
Aug 11 22:09:20 dist-test-slave-vxj2 krb5kdc[30880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486160, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:20.408411 31019 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:09:20.409591 31019 webserver.cc:460] Webserver started at http://127.28.47.129:42911/ using document root <none> and password file <none>
I20260811 22:09:20.410234 31019 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:20.410343 31019 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:20.410607 31019 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:20.412993 31019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "9f1cf3f6d31f427f8ce8030d53fde338"
format_stamp: "Formatted at 2026-08-11 22:09:20 on dist-test-slave-vxj2"
server_key: "b9f83d4ba7dcb3eb429466023bde0d6c"
server_key_iv: "5e56de270876d2715c76af62a29d1d36"
server_key_version: "encryptionkey@0"
I20260811 22:09:20.413770 31019 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "9f1cf3f6d31f427f8ce8030d53fde338"
format_stamp: "Formatted at 2026-08-11 22:09:20 on dist-test-slave-vxj2"
server_key: "b9f83d4ba7dcb3eb429466023bde0d6c"
server_key_iv: "5e56de270876d2715c76af62a29d1d36"
server_key_version: "encryptionkey@0"
I20260811 22:09:20.418994 31019 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:09:20.422210 31037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:20.423732 31019 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:09:20.423979 31019 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "9f1cf3f6d31f427f8ce8030d53fde338"
format_stamp: "Formatted at 2026-08-11 22:09:20 on dist-test-slave-vxj2"
server_key: "b9f83d4ba7dcb3eb429466023bde0d6c"
server_key_iv: "5e56de270876d2715c76af62a29d1d36"
server_key_version: "encryptionkey@0"
I20260811 22:09:20.424129 31019 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:20.451887 31019 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:20.456597 31019 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:20.457410 31019 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:20.459874 31019 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:20.459980 31019 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:20.460078 31019 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:20.460143 31019 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:09:20 dist-test-slave-vxj2 krb5kdc[30880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486160, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:20.473594 31019 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:40549
I20260811 22:09:20.473680 31173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:40549 every 8 connection(s)
I20260811 22:09:20.474689 31019 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:20.477695 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:20.461225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:46333 (local address 127.28.47.190:44257)
0811 22:09:20.461409 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:20.461413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:20.463280 (+ 1867us) server_negotiation.cc:407] Connection header received
0811 22:09:20.464838 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:20.464846 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:20.464925 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:20.465040 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:20.466685 (+ 1645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.467676 (+ 991us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.469194 (+ 1518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.469458 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.469606 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:20.473218 (+ 3612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:20.473225 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:20.473228 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:20.473270 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:20.475176 (+ 1906us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.475935 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.475939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.475941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.476043 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.476425 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.476429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.476431 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.476653 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:20.476772 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:20.477363 (+ 591us) server_negotiation.cc:299] Negotiation successful
0811 22:09:20.477517 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:09:20.478441 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 31019
I20260811 22:09:20.478586 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:09:20.478826 31047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:20.462286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:44257 (local address 127.28.47.129:46333)
0811 22:09:20.463076 (+ 790us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:20.463147 (+ 71us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:20.464544 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:20.465299 (+ 755us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:20.465307 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:20.465945 (+ 638us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:20.466516 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:20.466531 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.467843 (+ 1312us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.467848 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:20.469004 (+ 1156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:20.469015 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.469658 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.469662 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:20.469896 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:20.470703 (+ 807us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:20.470729 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:20.472968 (+ 2239us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:20.475391 (+ 2423us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:20.475397 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:20.475414 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:20.475814 (+ 400us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:20.476145 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:20.476149 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:20.476151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:20.476314 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:20.476774 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:20.476783 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:20.477054 (+ 271us) client_negotiation.cc:769] Sending connection context
0811 22:09:20.477318 (+ 264us) client_negotiation.cc:240] Negotiation successful
0811 22:09:20.477683 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":479,"thread_start_us":174,"threads_started":1}
I20260811 22:09:20.478931 28862 external_mini_cluster.cc:1428] Setting key 93d217618df699c168be4c2811f42746
I20260811 22:09:20.480517 31174 heartbeater.cc:347] Connected to a master server at 127.28.47.190:44257
I20260811 22:09:20.480769 31174 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:20.481467 31174 heartbeater.cc:511] Master 127.28.47.190:44257 requested a full tablet report, sending...
I20260811 22:09:20.483664 30933 ts_manager.cc:194] Registered new tserver with Master: 9f1cf3f6d31f427f8ce8030d53fde338 (127.28.47.129:40549)
I20260811 22:09:20.485483 30933 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:46333
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:09:20.495239 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:20.486348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:56919 (local address 127.28.47.190:44257)
0811 22:09:20.486908 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:20.486912 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:20.486930 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:09:20.486972 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:20.486975 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:20.487021 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:20.487117 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:20.487797 (+ 680us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.488715 (+ 918us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.489932 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.490162 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.490284 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:20.491562 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:20.491566 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:20.491571 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:20.491602 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:20.493417 (+ 1815us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.493911 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.493919 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.493921 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.494002 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.494308 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.494311 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.494313 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.494527 (+ 214us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:20.494645 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:20.494941 (+ 296us) server_negotiation.cc:299] Negotiation successful
0811 22:09:20.495065 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":465}
I20260811 22:09:20.495253 31181 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:20.484499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:44257 (local address 127.28.47.129:56919)
0811 22:09:20.485556 (+ 1057us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:20.485572 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:20.485679 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:20.487178 (+ 1499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:20.487183 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:20.487491 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:20.487644 (+ 153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:20.487653 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.489090 (+ 1437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.489095 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:20.489783 (+ 688us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:20.489794 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.490295 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.490299 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:20.490413 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:20.490999 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:20.491013 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:20.491400 (+ 387us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:20.493553 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:20.493556 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:20.493559 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:20.493788 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:20.494108 (+ 320us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:20.494112 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:20.494114 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:20.494207 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:20.494634 (+ 427us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:20.494637 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:20.494756 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 22:09:20.494876 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 22:09:20.495057 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":900,"thread_start_us":144,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:09:20.517457 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:44257
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:41763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:20.644644 31183 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:20.645031 31183 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:20.645140 31183 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:20.649931 31183 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:20.650036 31183 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:20.650110 31183 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:09:20.650156 31183 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:09:20.650226 31183 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:20.656504 31183 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:41763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.47.190:44257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31183
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:20.657918 31183 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:20.659114 31183 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:20.669713 31192 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:20.669718 31188 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:20.669700 31189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:20.669965 31183 server_base.cc:1034] running on GCE node
I20260811 22:09:20.670554 31183 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:20.671265 31183 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:20.672472 31183 hybrid_clock.cc:648] HybridClock initialized: now 1786486160672440 us; error 47 us; skew 500 ppm
Aug 11 22:09:20 dist-test-slave-vxj2 krb5kdc[30880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486160, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:20.675729 31183 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:09:20.676770 31183 webserver.cc:460] Webserver started at http://127.28.47.130:38955/ using document root <none> and password file <none>
I20260811 22:09:20.677477 31183 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:20.677549 31183 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:20.677811 31183 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:20.680225 31183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "b5576f14aca34946aefc1f726b3ad5f5"
format_stamp: "Formatted at 2026-08-11 22:09:20 on dist-test-slave-vxj2"
server_key: "bb49a7728ad1821fae245b7564af9b08"
server_key_iv: "439a001ee9a3e3486f2de7eabcb78d81"
server_key_version: "encryptionkey@0"
I20260811 22:09:20.680908 31183 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "b5576f14aca34946aefc1f726b3ad5f5"
format_stamp: "Formatted at 2026-08-11 22:09:20 on dist-test-slave-vxj2"
server_key: "bb49a7728ad1821fae245b7564af9b08"
server_key_iv: "439a001ee9a3e3486f2de7eabcb78d81"
server_key_version: "encryptionkey@0"
I20260811 22:09:20.685496 31183 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:09:20.688635 31199 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:20.690448 31183 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:20.690598 31183 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "b5576f14aca34946aefc1f726b3ad5f5"
format_stamp: "Formatted at 2026-08-11 22:09:20 on dist-test-slave-vxj2"
server_key: "bb49a7728ad1821fae245b7564af9b08"
server_key_iv: "439a001ee9a3e3486f2de7eabcb78d81"
server_key_version: "encryptionkey@0"
I20260811 22:09:20.690784 31183 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:20.710991 31183 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:20.715584 31183 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:20.715901 31183 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:20.717634 31183 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:20.717722 31183 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:20.717810 31183 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:20.717856 31183 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:20.729884 31183 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:39123
I20260811 22:09:20.729997 31341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:39123 every 8 connection(s)
Aug 11 22:09:20 dist-test-slave-vxj2 krb5kdc[30880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486160, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:20.731166 31183 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:20.735910 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:20.719058 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:57439 (local address 127.28.47.190:44257)
0811 22:09:20.719205 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:20.719209 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:20.720538 (+ 1329us) server_negotiation.cc:407] Connection header received
0811 22:09:20.722162 (+ 1624us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:20.722166 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:20.722238 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:20.722342 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:20.724388 (+ 2046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.725385 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.727617 (+ 2232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.728027 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.728181 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:20.731815 (+ 3634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:20.731820 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:20.731823 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:20.731857 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:20.733578 (+ 1721us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.734164 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.734167 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.734168 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.734270 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.734705 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.734709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.734710 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.734976 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:20.735074 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:20.735632 (+ 558us) server_negotiation.cc:299] Negotiation successful
0811 22:09:20.735759 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:09:20.736439 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 31183
I20260811 22:09:20.736569 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:09:20.736923 28862 external_mini_cluster.cc:1428] Setting key 91638d58a0fba835840e715f4e85b122
I20260811 22:09:20.736912 31211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:20.719658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:44257 (local address 127.28.47.130:57439)
0811 22:09:20.720334 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:20.720403 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:20.721870 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:20.722611 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:20.722626 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:20.723265 (+ 639us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:20.724132 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:20.724160 (+ 28us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.725607 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.725615 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:20.727416 (+ 1801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:20.727433 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.728255 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.728261 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:20.728483 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:20.729404 (+ 921us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:20.729425 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:20.731578 (+ 2153us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:20.733704 (+ 2126us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:20.733709 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:20.733722 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:20.734015 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:20.734400 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:20.734403 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:20.734404 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:20.734574 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:20.735093 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:20.735098 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:20.735364 (+ 266us) client_negotiation.cc:769] Sending connection context
0811 22:09:20.735545 (+ 181us) client_negotiation.cc:240] Negotiation successful
0811 22:09:20.735831 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":142,"threads_started":1}
I20260811 22:09:20.738444 31342 heartbeater.cc:347] Connected to a master server at 127.28.47.190:44257
I20260811 22:09:20.738628 31342 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:20.739166 31342 heartbeater.cc:511] Master 127.28.47.190:44257 requested a full tablet report, sending...
I20260811 22:09:20.740295 30935 ts_manager.cc:194] Registered new tserver with Master: b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130:39123)
I20260811 22:09:20.741114 30935 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:57439
I20260811 22:09:20.750192 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:20.741258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:33397 (local address 127.28.47.190:44257)
0811 22:09:20.741393 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:20.741399 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:20.741767 (+ 368us) server_negotiation.cc:407] Connection header received
0811 22:09:20.741914 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:20.741917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:20.741967 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:20.742056 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:20.742609 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.743342 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.744246 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.744510 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.744599 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:20.746254 (+ 1655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:20.746257 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:20.746259 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:20.746282 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:20.748051 (+ 1769us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.748676 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.748680 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.748681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.748766 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.749096 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.749098 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.749100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.749446 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:20.749537 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:20.749947 (+ 410us) server_negotiation.cc:299] Negotiation successful
0811 22:09:20.750063 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:09:20.750312 31350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:20.741174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:44257 (local address 127.28.47.130:33397)
0811 22:09:20.741672 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:20.741684 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:20.741818 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:20.742073 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:20.742076 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:20.742306 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:20.742456 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:20.742468 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.743484 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.743487 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:20.744130 (+ 643us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:20.744139 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.744651 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.744658 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:20.744790 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:20.745524 (+ 734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:20.745544 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:20.746112 (+ 568us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:20.748208 (+ 2096us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:20.748214 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:20.748219 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:20.748541 (+ 322us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:20.748881 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:20.748886 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:20.748888 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:20.748997 (+ 109us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:20.749565 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:20.749568 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:20.749705 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 22:09:20.749836 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 22:09:20.750036 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":137,"threads_started":1}
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:09:20.773828 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:44257
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:41763
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:20.899200 31353 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:20.899547 31353 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:20.899603 31353 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:20.904088 31353 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:20.904191 31353 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:20.904239 31353 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260811 22:09:20.904258 31353 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:09:20.904296 31353 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:20.910352 31353 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:41763
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.47.190:44257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31353
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:20.911681 31353 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:20.912891 31353 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:20.924103 31360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:20.924103 31359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:20.924108 31363 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:20.924556 31353 server_base.cc:1034] running on GCE node
I20260811 22:09:20.925269 31353 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:20.926100 31353 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:20.927337 31353 hybrid_clock.cc:648] HybridClock initialized: now 1786486160927319 us; error 35 us; skew 500 ppm
Aug 11 22:09:20 dist-test-slave-vxj2 krb5kdc[30880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486160, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:20.930722 31353 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:09:20.931772 31353 webserver.cc:460] Webserver started at http://127.28.47.131:40225/ using document root <none> and password file <none>
I20260811 22:09:20.932464 31353 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:20.932571 31353 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:20.932847 31353 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:20.935293 31353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "0de1775bd76b49fdb7eac8c179567356"
format_stamp: "Formatted at 2026-08-11 22:09:20 on dist-test-slave-vxj2"
server_key: "b5714e59e24fb553c16848dbcc8abefb"
server_key_iv: "8c6b52f41b8b91eb7fb8d93615889d9f"
server_key_version: "encryptionkey@0"
I20260811 22:09:20.936085 31353 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "0de1775bd76b49fdb7eac8c179567356"
format_stamp: "Formatted at 2026-08-11 22:09:20 on dist-test-slave-vxj2"
server_key: "b5714e59e24fb553c16848dbcc8abefb"
server_key_iv: "8c6b52f41b8b91eb7fb8d93615889d9f"
server_key_version: "encryptionkey@0"
I20260811 22:09:20.941131 31353 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:09:20.944701 31374 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:20.946504 31353 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:09:20.946671 31353 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "0de1775bd76b49fdb7eac8c179567356"
format_stamp: "Formatted at 2026-08-11 22:09:20 on dist-test-slave-vxj2"
server_key: "b5714e59e24fb553c16848dbcc8abefb"
server_key_iv: "8c6b52f41b8b91eb7fb8d93615889d9f"
server_key_version: "encryptionkey@0"
I20260811 22:09:20.946836 31353 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:20.961081 31353 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:20.964529 31353 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:20.964766 31353 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:20.966483 31353 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:20.966573 31353 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:20.966651 31353 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:20.966703 31353 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:09:20 dist-test-slave-vxj2 krb5kdc[30880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486160, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:20.978277 31353 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:39377
I20260811 22:09:20.978365 31511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:39377 every 8 connection(s)
I20260811 22:09:20.979526 31353 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:20.982247 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 31353
I20260811 22:09:20.982394 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:09:20.982762 28862 external_mini_cluster.cc:1428] Setting key 9f5b6473c8659f79eb4262f1e6a094d1
I20260811 22:09:20.983024 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:20.967617 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:48733 (local address 127.28.47.190:44257)
0811 22:09:20.967797 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:20.967804 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:20.968766 (+ 962us) server_negotiation.cc:407] Connection header received
0811 22:09:20.970167 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:20.970171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:20.970248 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:20.970389 (+ 141us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:20.972251 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.973131 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.974464 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.974806 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.974943 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:20.978212 (+ 3269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:20.978217 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:20.978219 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:20.978257 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:20.979951 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.980680 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.980685 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.980687 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.980800 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.981238 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.981242 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.981244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.981528 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:20.981672 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:20.982450 (+ 778us) server_negotiation.cc:299] Negotiation successful
0811 22:09:20.982739 (+ 289us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:09:20.983843 31383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:20.968065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:44257 (local address 127.28.47.131:48733)
0811 22:09:20.968587 (+ 522us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:20.968642 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:20.969922 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:20.970604 (+ 682us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:20.970614 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:20.971197 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:20.972026 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:20.972053 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.973302 (+ 1249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.973307 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:20.974292 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:20.974303 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.974981 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.974985 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:20.975203 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:20.975952 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:20.975983 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:20.977989 (+ 2006us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:20.980100 (+ 2111us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:20.980107 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:20.980123 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:20.980543 (+ 420us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:20.980906 (+ 363us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:20.980910 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:20.980912 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:20.981087 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:20.981666 (+ 579us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:20.981673 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:20.982037 (+ 364us) client_negotiation.cc:769] Sending connection context
0811 22:09:20.982307 (+ 270us) client_negotiation.cc:240] Negotiation successful
0811 22:09:20.982741 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":161,"threads_started":1}
I20260811 22:09:20.985116 31512 heartbeater.cc:347] Connected to a master server at 127.28.47.190:44257
I20260811 22:09:20.985308 31512 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:20.985832 31512 heartbeater.cc:511] Master 127.28.47.190:44257 requested a full tablet report, sending...
I20260811 22:09:20.986932 30936 ts_manager.cc:194] Registered new tserver with Master: 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131:39377)
I20260811 22:09:20.987706 30936 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:48733
I20260811 22:09:20.995105 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:20.987572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:44257 (local address 127.28.47.131:59397)
0811 22:09:20.987898 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:20.987913 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:20.988043 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:20.988279 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:20.988282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:20.988531 (+ 249us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:20.988635 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:20.988641 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.989671 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.989674 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:20.990224 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:20.990232 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.990769 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.990772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:20.990852 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:20.991458 (+ 606us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:20.991470 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:20.991814 (+ 344us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:20.993658 (+ 1844us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:20.993661 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:20.993662 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:20.993879 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:20.994192 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:20.994194 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:20.994195 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:20.994272 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:20.994654 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:20.994656 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:20.994742 (+ 86us) client_negotiation.cc:769] Sending connection context
0811 22:09:20.994822 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:09:20.994951 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":112,"threads_started":1}
I20260811 22:09:20.995162 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:20.987691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:59397 (local address 127.28.47.190:44257)
0811 22:09:20.987813 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:20.987817 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:20.987993 (+ 176us) server_negotiation.cc:407] Connection header received
0811 22:09:20.988126 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:20.988128 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:20.988163 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:20.988236 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:20.988756 (+ 520us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.989551 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.990387 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:20.990630 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:20.990731 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:20.991958 (+ 1227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:20.991961 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:20.991964 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:20.992017 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:20.993555 (+ 1538us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.993993 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.993996 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.993997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.994083 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:20.994364 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:20.994366 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:20.994367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:20.994570 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:20.994652 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:20.994894 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 22:09:20.995002 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260811 22:09:20.997478 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:09:21 dist-test-slave-vxj2 krb5kdc[30880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:09:21 dist-test-slave-vxj2 krb5kdc[30880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486161, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:21.027784 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.018115 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33518 (local address 127.28.47.190:44257)
0811 22:09:21.018293 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:21.018298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:21.018314 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:09:21.018358 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:21.018361 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:21.018403 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:21.018482 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:21.019299 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.019903 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.020872 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.021109 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.021348 (+ 239us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.024189 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:21.024193 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:21.024197 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:21.024234 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:21.025947 (+ 1713us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:21.026458 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:21.026462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:21.026463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:21.026547 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:21.026849 (+ 302us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:21.026853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:21.026854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:21.027063 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:21.027209 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:21.027471 (+ 262us) server_negotiation.cc:299] Negotiation successful
0811 22:09:21.027601 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:09:21.031553 30933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:09:21.034864 30933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:21.052201 31535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.044655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:39377 (local address 127.0.0.1:46814)
0811 22:09:21.045029 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:21.045080 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:21.045267 (+ 187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:21.047054 (+ 1787us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:21.047057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:21.047075 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:21.047211 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.047229 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.048724 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.048728 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.050414 (+ 1686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.050423 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.051491 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.051495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.051687 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.051743 (+ 56us) client_negotiation.cc:769] Sending connection context
0811 22:09:21.051843 (+ 100us) client_negotiation.cc:240] Negotiation successful
0811 22:09:21.052016 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":176,"mutex_wait_us":37,"thread_start_us":201,"threads_started":1}
I20260811 22:09:21.052294 31538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.045673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53668 (local address 127.28.47.130:39123)
0811 22:09:21.046172 (+ 499us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:21.046190 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:21.046213 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:09:21.046279 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:21.046282 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:21.046531 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:21.046766 (+ 235us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:21.047321 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.048220 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.050289 (+ 2069us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.051234 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.051352 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.051548 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:21.052005 (+ 457us) server_negotiation.cc:299] Negotiation successful
0811 22:09:21.052118 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":143,"threads_started":1}
I20260811 22:09:21.052484 31539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.045828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46814 (local address 127.28.47.131:39377)
0811 22:09:21.046522 (+ 694us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:21.046534 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:21.046561 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:09:21.046683 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:21.046689 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:21.046889 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:21.047060 (+ 171us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:21.047354 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.048571 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.050588 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.051334 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.051993 (+ 659us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.052157 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:21.052210 (+ 53us) server_negotiation.cc:299] Negotiation successful
0811 22:09:21.052332 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":565,"thread_start_us":132,"threads_started":1}
I20260811 22:09:21.052866 31537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.045459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:40549 (local address 127.0.0.1:45318)
0811 22:09:21.045691 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:21.045706 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:21.045831 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:21.047230 (+ 1399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:21.047234 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:21.047251 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:21.047349 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.047355 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.048769 (+ 1414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.048772 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.050198 (+ 1426us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.050210 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.051705 (+ 1495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.051708 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.051825 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.051850 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:09:21.051904 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 22:09:21.052743 (+ 839us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":102,"threads_started":1}
I20260811 22:09:21.053287 31536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.044668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:39123 (local address 127.0.0.1:53668)
0811 22:09:21.045216 (+ 548us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:21.045230 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:21.045332 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:21.046973 (+ 1641us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:21.046979 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:21.047002 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:21.047184 (+ 182us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.047193 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.048365 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.048368 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.050095 (+ 1727us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.050109 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.051621 (+ 1512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.051625 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.051764 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.051801 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:09:21.051897 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 22:09:21.053117 (+ 1220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":455,"mutex_wait_us":1171,"thread_start_us":76,"threads_started":1}
I20260811 22:09:21.053927 31540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.046384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45318 (local address 127.28.47.129:40549)
0811 22:09:21.046744 (+ 360us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:21.046750 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:21.046773 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:09:21.046858 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:21.046863 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:21.047044 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:21.047185 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:21.047503 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.048631 (+ 1128us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.050369 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.051332 (+ 963us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.051454 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.053607 (+ 2153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:21.053676 (+ 69us) server_negotiation.cc:299] Negotiation successful
0811 22:09:21.053796 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":212,"thread_start_us":110,"threads_started":1}
I20260811 22:09:21.056170 31094 tablet_service.cc:1467] Processing CreateTablet for tablet 01d7b950e7bf4cf693530ce1d9122c05 (DEFAULT_TABLE table=test-table [id=49416707247b41e4a00703c35e888fb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:21.056015 31434 tablet_service.cc:1467] Processing CreateTablet for tablet 01d7b950e7bf4cf693530ce1d9122c05 (DEFAULT_TABLE table=test-table [id=49416707247b41e4a00703c35e888fb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:21.056228 31261 tablet_service.cc:1467] Processing CreateTablet for tablet 01d7b950e7bf4cf693530ce1d9122c05 (DEFAULT_TABLE table=test-table [id=49416707247b41e4a00703c35e888fb5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:21.057281 31094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d7b950e7bf4cf693530ce1d9122c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:21.057291 31434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d7b950e7bf4cf693530ce1d9122c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:21.057286 31261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 01d7b950e7bf4cf693530ce1d9122c05. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:21.064179 31541 tablet_bootstrap.cc:492] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338: Bootstrap starting.
I20260811 22:09:21.065837 31543 tablet_bootstrap.cc:492] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356: Bootstrap starting.
I20260811 22:09:21.066901 31542 tablet_bootstrap.cc:492] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5: Bootstrap starting.
I20260811 22:09:21.067360 31541 tablet_bootstrap.cc:654] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:21.068429 31543 tablet_bootstrap.cc:654] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:21.068405 31541 log.cc:824] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:21.068988 31542 tablet_bootstrap.cc:654] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:21.069334 31543 log.cc:824] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:21.069868 31542 log.cc:824] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:21.070983 31541 tablet_bootstrap.cc:492] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338: No bootstrap required, opened a new log
I20260811 22:09:21.071223 31541 ts_tablet_manager.cc:1397] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:09:21.071468 31543 tablet_bootstrap.cc:492] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356: No bootstrap required, opened a new log
I20260811 22:09:21.071671 31543 ts_tablet_manager.cc:1397] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:09:21.071791 31542 tablet_bootstrap.cc:492] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5: No bootstrap required, opened a new log
I20260811 22:09:21.071990 31542 ts_tablet_manager.cc:1397] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5: Time spent bootstrapping tablet: real 0.005s user 0.004s sys 0.000s
I20260811 22:09:21.075413 31542 raft_consensus.cc:348] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } }
I20260811 22:09:21.075700 31542 raft_consensus.cc:374] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:21.075766 31542 raft_consensus.cc:729] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5576f14aca34946aefc1f726b3ad5f5, State: Initialized, Role: FOLLOWER
I20260811 22:09:21.075758 31543 raft_consensus.cc:348] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } }
I20260811 22:09:21.075834 31541 raft_consensus.cc:348] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } }
I20260811 22:09:21.076040 31541 raft_consensus.cc:374] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:21.076040 31543 raft_consensus.cc:374] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:21.076099 31543 raft_consensus.cc:729] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0de1775bd76b49fdb7eac8c179567356, State: Initialized, Role: FOLLOWER
I20260811 22:09:21.076109 31541 raft_consensus.cc:729] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1cf3f6d31f427f8ce8030d53fde338, State: Initialized, Role: FOLLOWER
I20260811 22:09:21.076282 31542 consensus_queue.cc:260] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } }
I20260811 22:09:21.076548 31543 consensus_queue.cc:260] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } }
I20260811 22:09:21.076548 31541 consensus_queue.cc:260] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } }
I20260811 22:09:21.077207 31342 heartbeater.cc:503] Master 127.28.47.190:44257 was elected leader, sending a full tablet report...
I20260811 22:09:21.077517 31542 ts_tablet_manager.cc:1428] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260811 22:09:21.077543 31543 ts_tablet_manager.cc:1428] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:09:21.077909 31512 heartbeater.cc:503] Master 127.28.47.190:44257 was elected leader, sending a full tablet report...
I20260811 22:09:21.078310 31541 ts_tablet_manager.cc:1428] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:09:21.078626 31174 heartbeater.cc:503] Master 127.28.47.190:44257 was elected leader, sending a full tablet report...
I20260811 22:09:21.108839 31549 raft_consensus.cc:482] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:21.109172 31549 raft_consensus.cc:504] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } }
I20260811 22:09:21.111475 31549 leader_election.cc:288] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131:39377), b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130:39123)
I20260811 22:09:21.115971 31539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.110995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:38705 (local address 127.28.47.131:39377)
0811 22:09:21.111256 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:21.111262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:21.111542 (+ 280us) server_negotiation.cc:407] Connection header received
0811 22:09:21.111819 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:21.111822 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:21.111872 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:21.111955 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:21.112380 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.112902 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.114463 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.115094 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.115194 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.115252 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:21.115684 (+ 432us) server_negotiation.cc:299] Negotiation successful
0811 22:09:21.115783 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159}
I20260811 22:09:21.116178 31538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.111117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:54757 (local address 127.28.47.130:39123)
0811 22:09:21.111263 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:21.111269 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:21.111462 (+ 193us) server_negotiation.cc:407] Connection header received
0811 22:09:21.111630 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:21.111636 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:21.111698 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:21.111777 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:21.112367 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.112903 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.114730 (+ 1827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.115345 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.115558 (+ 213us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.115683 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:21.115946 (+ 263us) server_negotiation.cc:299] Negotiation successful
0811 22:09:21.116045 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:09:21.116253 31553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.110864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:39123 (local address 127.28.47.129:54757)
0811 22:09:21.111362 (+ 498us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:21.111376 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:21.111523 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:21.111945 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:21.111948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:21.111973 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:21.112213 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.112222 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.113028 (+ 806us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.113033 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.114541 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.114557 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.115579 (+ 1022us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.115585 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.115741 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.115781 (+ 40us) client_negotiation.cc:769] Sending connection context
0811 22:09:21.115929 (+ 148us) client_negotiation.cc:240] Negotiation successful
0811 22:09:21.116025 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":414,"mutex_wait_us":78,"spinlock_wait_cycles":4736,"thread_start_us":116,"threads_started":1}
I20260811 22:09:21.117056 31285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d7b950e7bf4cf693530ce1d9122c05" candidate_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5576f14aca34946aefc1f726b3ad5f5" is_pre_election: true
I20260811 22:09:21.117470 31285 raft_consensus.cc:2454] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1cf3f6d31f427f8ce8030d53fde338 in term 0.
I20260811 22:09:21.118196 31552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.110863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:39377 (local address 127.28.47.129:38705)
0811 22:09:21.111450 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:21.111467 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:21.111590 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:21.111978 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:21.111981 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:21.111998 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:21.112255 (+ 257us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.112268 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.113026 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.113030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.114303 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.114317 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.115360 (+ 1043us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.115364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.115504 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.115541 (+ 37us) client_negotiation.cc:769] Sending connection context
0811 22:09:21.116565 (+ 1024us) client_negotiation.cc:240] Negotiation successful
0811 22:09:21.116650 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":506,"mutex_wait_us":70,"thread_start_us":96,"threads_started":1}
I20260811 22:09:21.118240 31039 leader_election.cc:302] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1cf3f6d31f427f8ce8030d53fde338, b5576f14aca34946aefc1f726b3ad5f5; no voters:
I20260811 22:09:21.118636 31549 raft_consensus.cc:2781] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:21.118747 31549 raft_consensus.cc:482] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:21.118789 31549 raft_consensus.cc:3037] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:21.118918 31455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d7b950e7bf4cf693530ce1d9122c05" candidate_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0de1775bd76b49fdb7eac8c179567356" is_pre_election: true
I20260811 22:09:21.119237 31455 raft_consensus.cc:2454] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1cf3f6d31f427f8ce8030d53fde338 in term 0.
I20260811 22:09:21.120270 31549 raft_consensus.cc:504] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } }
I20260811 22:09:21.120747 31549 leader_election.cc:288] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [CANDIDATE]: Term 1 election: Requested vote from peers 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131:39377), b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130:39123)
I20260811 22:09:21.121289 31455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d7b950e7bf4cf693530ce1d9122c05" candidate_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0de1775bd76b49fdb7eac8c179567356"
I20260811 22:09:21.121501 31455 raft_consensus.cc:3037] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:21.123243 31455 raft_consensus.cc:2454] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1cf3f6d31f427f8ce8030d53fde338 in term 1.
I20260811 22:09:21.123644 31285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "01d7b950e7bf4cf693530ce1d9122c05" candidate_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5576f14aca34946aefc1f726b3ad5f5"
I20260811 22:09:21.123710 31040 leader_election.cc:302] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0de1775bd76b49fdb7eac8c179567356, 9f1cf3f6d31f427f8ce8030d53fde338; no voters:
I20260811 22:09:21.123818 31285 raft_consensus.cc:3037] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:21.124033 31549 raft_consensus.cc:2781] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:21.124370 31549 raft_consensus.cc:686] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1cf3f6d31f427f8ce8030d53fde338, State: Running, Role: LEADER
I20260811 22:09:21.124797 31549 consensus_queue.cc:237] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } }
I20260811 22:09:21.125388 31285 raft_consensus.cc:2454] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1cf3f6d31f427f8ce8030d53fde338 in term 1.
I20260811 22:09:21.129256 30935 catalog_manager.cc:5495] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1cf3f6d31f427f8ce8030d53fde338 (127.28.47.129). New cstate: current_term: 1 leader_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } health_report { overall_health: HEALTHY } } }
Aug 11 22:09:21 dist-test-slave-vxj2 krb5kdc[30880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486160, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:21.190001 30993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.179428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33528 (local address 127.28.47.190:44257)
0811 22:09:21.179608 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:21.179612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:21.179911 (+ 299us) server_negotiation.cc:407] Connection header received
0811 22:09:21.180110 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:21.180113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:21.180159 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:21.180249 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:21.181206 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.181833 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.182692 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.182919 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.183014 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.185890 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:21.185893 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:21.185895 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:21.185921 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:21.187713 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:21.188364 (+ 651us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:21.188367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:21.188368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:21.188461 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:21.188872 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:21.188883 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:21.188887 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:21.189120 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:21.189270 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:21.189598 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 22:09:21.189819 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":78,"server-negotiator.queue_time_us":86}
I20260811 22:09:21.190003 31561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.179377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:44257 (local address 127.0.0.1:33528)
0811 22:09:21.179741 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:21.179759 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:21.179960 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:21.180342 (+ 382us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:21.180345 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:21.180806 (+ 461us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:21.180996 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.181007 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.181976 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.181979 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.182571 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.182580 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.183055 (+ 475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.183058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.183128 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.183783 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:21.183796 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:21.185647 (+ 1851us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:21.187899 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:21.187915 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:21.187926 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:21.188240 (+ 314us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:21.188580 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:21.188583 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:21.188585 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:21.188690 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:21.189272 (+ 582us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:21.189279 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:21.189426 (+ 147us) client_negotiation.cc:769] Sending connection context
0811 22:09:21.189538 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:09:21.189730 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":148,"threads_started":1}
I20260811 22:09:21.195132 30935 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:33528:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:09:21.196117 30935 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:21.201648 31094 tablet_service.cc:1467] Processing CreateTablet for tablet 370cf03c31824d7c9355248363d6b5a6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ad7198a55e5342eab7f45ab555de9d6d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:09:21.201762 31261 tablet_service.cc:1467] Processing CreateTablet for tablet 370cf03c31824d7c9355248363d6b5a6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ad7198a55e5342eab7f45ab555de9d6d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:09:21.202558 31094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370cf03c31824d7c9355248363d6b5a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:21.202558 31261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370cf03c31824d7c9355248363d6b5a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:21.202143 31434 tablet_service.cc:1467] Processing CreateTablet for tablet 370cf03c31824d7c9355248363d6b5a6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ad7198a55e5342eab7f45ab555de9d6d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:09:21.202857 31434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 370cf03c31824d7c9355248363d6b5a6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:21.205672 31543 tablet_bootstrap.cc:492] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356: Bootstrap starting.
I20260811 22:09:21.205765 31541 tablet_bootstrap.cc:492] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338: Bootstrap starting.
I20260811 22:09:21.205765 31542 tablet_bootstrap.cc:492] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5: Bootstrap starting.
I20260811 22:09:21.206624 31543 tablet_bootstrap.cc:654] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:21.207617 31542 tablet_bootstrap.cc:654] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:21.207617 31541 tablet_bootstrap.cc:654] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:21.209116 31541 tablet_bootstrap.cc:492] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338: No bootstrap required, opened a new log
I20260811 22:09:21.209117 31542 tablet_bootstrap.cc:492] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5: No bootstrap required, opened a new log
I20260811 22:09:21.209244 31542 ts_tablet_manager.cc:1397] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260811 22:09:21.209244 31541 ts_tablet_manager.cc:1397] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338: Time spent bootstrapping tablet: real 0.004s user 0.003s sys 0.000s
I20260811 22:09:21.209749 31542 raft_consensus.cc:348] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.209749 31541 raft_consensus.cc:348] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.209882 31541 raft_consensus.cc:374] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:21.209882 31542 raft_consensus.cc:374] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:21.209919 31541 raft_consensus.cc:729] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1cf3f6d31f427f8ce8030d53fde338, State: Initialized, Role: FOLLOWER
I20260811 22:09:21.209928 31542 raft_consensus.cc:729] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5576f14aca34946aefc1f726b3ad5f5, State: Initialized, Role: FOLLOWER
I20260811 22:09:21.210067 31541 consensus_queue.cc:260] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.210067 31542 consensus_queue.cc:260] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.210345 31542 ts_tablet_manager.cc:1428] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:09:21.210330 31549 tablet_replica.cc:439] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } }
I20260811 22:09:21.210366 31548 tablet_replica.cc:439] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } }
I20260811 22:09:21.210484 31549 tablet_replica.cc:442] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:21.210518 31548 tablet_replica.cc:442] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:21.210853 31541 ts_tablet_manager.cc:1428] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:09:21.211390 31543 tablet_bootstrap.cc:492] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356: No bootstrap required, opened a new log
I20260811 22:09:21.211503 31543 ts_tablet_manager.cc:1397] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260811 22:09:21.212102 31543 raft_consensus.cc:348] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.212245 31543 raft_consensus.cc:374] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:21.212282 31543 raft_consensus.cc:729] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0de1775bd76b49fdb7eac8c179567356, State: Initialized, Role: FOLLOWER
I20260811 22:09:21.212461 31543 consensus_queue.cc:260] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.212780 31543 ts_tablet_manager.cc:1428] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:09:21.212790 31551 tablet_replica.cc:439] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } }
I20260811 22:09:21.213038 31551 tablet_replica.cc:442] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356: This TxnStatusTablet replica's current role is: FOLLOWER
W20260811 22:09:21.226485 31175 tablet.cc:2369] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:09:21.231073 31514 tablet.cc:2369] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:09:21.232517 31343 tablet.cc:2369] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:09:21.271152 31551 raft_consensus.cc:482] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:21.271315 31551 raft_consensus.cc:504] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.272591 31551 leader_election.cc:288] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1cf3f6d31f427f8ce8030d53fde338 (127.28.47.129:40549), b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130:39123)
I20260811 22:09:21.277716 31540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.273210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:54415 (local address 127.28.47.129:40549)
0811 22:09:21.273435 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:21.273442 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:21.273461 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:09:21.273529 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:21.273533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:21.273603 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:21.273703 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:21.274192 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.274998 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.276492 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.277059 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.277224 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.277284 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:21.277487 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 22:09:21.277573 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260811 22:09:21.277722 31383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.272968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:40549 (local address 127.28.47.131:54415)
0811 22:09:21.273224 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:21.273242 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:21.273356 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:21.273865 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:21.273872 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:21.273899 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:21.274039 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.274048 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.275117 (+ 1069us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.275123 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.276347 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.276359 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.277254 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.277257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.277343 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.277366 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:09:21.277461 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 22:09:21.277555 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"mutex_wait_us":31}
I20260811 22:09:21.278019 31538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.273024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:46325 (local address 127.28.47.130:39123)
0811 22:09:21.273211 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:21.273217 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:21.274171 (+ 954us) server_negotiation.cc:407] Connection header received
0811 22:09:21.274289 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:21.274292 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:21.274336 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:21.274401 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:21.274782 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.275435 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.276783 (+ 1348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.277404 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.277556 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.277602 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:21.277789 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 22:09:21.277886 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:09:21.278502 31120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370cf03c31824d7c9355248363d6b5a6" candidate_uuid: "0de1775bd76b49fdb7eac8c179567356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" is_pre_election: true
I20260811 22:09:21.278636 31566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.272891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:39123 (local address 127.28.47.131:46325)
0811 22:09:21.274056 (+ 1165us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:21.274073 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:21.274181 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:21.274458 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:21.274461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:21.274480 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:21.274615 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.274621 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.275565 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.275568 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.276663 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.276670 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.277560 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.277564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.277652 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.277674 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:09:21.277754 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:09:21.278494 (+ 740us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":1,"thread_start_us":189,"threads_started":1}
I20260811 22:09:21.278774 31120 raft_consensus.cc:2454] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0de1775bd76b49fdb7eac8c179567356 in term 0.
I20260811 22:09:21.279227 31285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370cf03c31824d7c9355248363d6b5a6" candidate_uuid: "0de1775bd76b49fdb7eac8c179567356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5576f14aca34946aefc1f726b3ad5f5" is_pre_election: true
I20260811 22:09:21.279412 31285 raft_consensus.cc:2454] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0de1775bd76b49fdb7eac8c179567356 in term 0.
I20260811 22:09:21.279399 31376 leader_election.cc:302] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0de1775bd76b49fdb7eac8c179567356, 9f1cf3f6d31f427f8ce8030d53fde338; no voters:
I20260811 22:09:21.279783 31551 raft_consensus.cc:2781] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:21.279861 31551 raft_consensus.cc:482] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:21.279907 31551 raft_consensus.cc:3037] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:21.281337 31551 raft_consensus.cc:504] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.281788 31551 leader_election.cc:288] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1cf3f6d31f427f8ce8030d53fde338 (127.28.47.129:40549), b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130:39123)
I20260811 22:09:21.282195 31120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370cf03c31824d7c9355248363d6b5a6" candidate_uuid: "0de1775bd76b49fdb7eac8c179567356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1cf3f6d31f427f8ce8030d53fde338"
I20260811 22:09:21.282387 31120 raft_consensus.cc:3037] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:21.282387 31285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "370cf03c31824d7c9355248363d6b5a6" candidate_uuid: "0de1775bd76b49fdb7eac8c179567356" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5576f14aca34946aefc1f726b3ad5f5"
I20260811 22:09:21.282757 31285 raft_consensus.cc:3037] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:21.283795 31120 raft_consensus.cc:2454] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0de1775bd76b49fdb7eac8c179567356 in term 1.
I20260811 22:09:21.284073 31285 raft_consensus.cc:2454] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0de1775bd76b49fdb7eac8c179567356 in term 1.
I20260811 22:09:21.284235 31376 leader_election.cc:302] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0de1775bd76b49fdb7eac8c179567356, 9f1cf3f6d31f427f8ce8030d53fde338; no voters:
I20260811 22:09:21.284472 31551 raft_consensus.cc:2781] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:21.284731 31551 raft_consensus.cc:686] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [term 1 LEADER]: Becoming Leader. State: Replica: 0de1775bd76b49fdb7eac8c179567356, State: Running, Role: LEADER
I20260811 22:09:21.285265 31551 consensus_queue.cc:237] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.288043 31567 tablet_replica.cc:439] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356: TxnStatusTablet state changed. Reason: New leader 0de1775bd76b49fdb7eac8c179567356. Latest consensus state: current_term: 1 leader_uuid: "0de1775bd76b49fdb7eac8c179567356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } }
I20260811 22:09:21.288195 31567 tablet_replica.cc:442] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:09:21.288851 31569 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:09:21.289100 31569 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:09:21.289352 30935 catalog_manager.cc:5495] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131). New cstate: current_term: 1 leader_uuid: "0de1775bd76b49fdb7eac8c179567356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } health_report { overall_health: HEALTHY } } }
I20260811 22:09:21.324011 31561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.319062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:39377 (local address 127.0.0.1:46822)
0811 22:09:21.319386 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:21.319408 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:21.319524 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:21.319972 (+ 448us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:21.319975 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:21.319983 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:09:21.320096 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.320101 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.321258 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.321261 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.321917 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.321929 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.322558 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.322560 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.322637 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.322700 (+ 63us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:09:21.323648 (+ 948us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:09:21.323668 (+ 20us) client_negotiation.cc:769] Sending connection context
0811 22:09:21.323739 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:09:21.323839 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166}
I20260811 22:09:21.324041 31539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.319278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46822 (local address 127.28.47.131:39377)
0811 22:09:21.319449 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:21.319455 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:21.319478 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:09:21.319708 (+ 230us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:21.319712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:21.319793 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:21.319915 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:21.320242 (+ 327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.321041 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.322063 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.322418 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.322542 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.322872 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:21.323503 (+ 631us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:21.323640 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:21.323776 (+ 136us) server_negotiation.cc:299] Negotiation successful
0811 22:09:21.323850 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:09:21.330673 31285 raft_consensus.cc:1261] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Refusing update from remote peer 0de1775bd76b49fdb7eac8c179567356: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:21.330673 31120 raft_consensus.cc:1261] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Refusing update from remote peer 0de1775bd76b49fdb7eac8c179567356: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:21.331519 31567 consensus_queue.cc:1035] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:21.331928 31551 consensus_queue.cc:1035] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:21.336131 31549 tablet_replica.cc:439] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338: TxnStatusTablet state changed. Reason: New leader 0de1775bd76b49fdb7eac8c179567356. Latest consensus state: current_term: 1 leader_uuid: "0de1775bd76b49fdb7eac8c179567356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } }
I20260811 22:09:21.336305 31549 tablet_replica.cc:442] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:21.336356 31548 tablet_replica.cc:439] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5: TxnStatusTablet state changed. Reason: New leader 0de1775bd76b49fdb7eac8c179567356. Latest consensus state: current_term: 1 leader_uuid: "0de1775bd76b49fdb7eac8c179567356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } }
I20260811 22:09:21.336529 31548 tablet_replica.cc:442] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:21.341756 31551 tablet_replica.cc:439] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0de1775bd76b49fdb7eac8c179567356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } }
I20260811 22:09:21.341917 31551 tablet_replica.cc:442] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:09:21.342051 31551 tablet_replica.cc:439] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0de1775bd76b49fdb7eac8c179567356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } }
I20260811 22:09:21.342127 31551 tablet_replica.cc:442] T 370cf03c31824d7c9355248363d6b5a6 P 0de1775bd76b49fdb7eac8c179567356: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:09:21.342217 31574 mvcc.cc:205] Tried to move back new op lower bound from 7317447316801990656 to 7317447316628283392. Current Snapshot: MvccSnapshot[applied={T|T < 7317447316801990656}]
I20260811 22:09:21.342250 31548 tablet_replica.cc:439] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0de1775bd76b49fdb7eac8c179567356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } }
I20260811 22:09:21.342391 31548 tablet_replica.cc:442] T 370cf03c31824d7c9355248363d6b5a6 P b5576f14aca34946aefc1f726b3ad5f5: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:21.342708 31549 tablet_replica.cc:439] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0de1775bd76b49fdb7eac8c179567356" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } } }
I20260811 22:09:21.342831 31549 tablet_replica.cc:442] T 370cf03c31824d7c9355248363d6b5a6 P 9f1cf3f6d31f427f8ce8030d53fde338: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:21.344203 31575 mvcc.cc:205] Tried to move back new op lower bound from 7317447316801990656 to 7317447316628283392. Current Snapshot: MvccSnapshot[applied={T|T < 7317447316801990656}]
I20260811 22:09:21.344229 31576 mvcc.cc:205] Tried to move back new op lower bound from 7317447316801990656 to 7317447316628283392. Current Snapshot: MvccSnapshot[applied={T|T < 7317447316801990656}]
I20260811 22:09:21.356630 31540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.352153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45324 (local address 127.28.47.129:40549)
0811 22:09:21.352310 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:21.352314 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:21.352331 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:09:21.352543 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:21.352546 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:21.352616 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:21.352737 (+ 121us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:21.353036 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.353731 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.354634 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.354905 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.355017 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.355316 (+ 299us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:21.355817 (+ 501us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:21.356295 (+ 478us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:21.356371 (+ 76us) server_negotiation.cc:299] Negotiation successful
0811 22:09:21.356432 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:09:21.370575 31539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.366133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:58985 (local address 127.28.47.131:39377)
0811 22:09:21.366295 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:21.366300 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:21.366549 (+ 249us) server_negotiation.cc:407] Connection header received
0811 22:09:21.366700 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:21.366702 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:21.366750 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:21.366862 (+ 112us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:21.367211 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.367724 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.369266 (+ 1542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.369500 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.369602 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.370077 (+ 475us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:21.370165 (+ 88us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:21.370232 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:21.370397 (+ 165us) server_negotiation.cc:299] Negotiation successful
0811 22:09:21.370455 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:09:21.370666 31588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.366035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:39377 (local address 127.28.47.129:58985)
0811 22:09:21.366423 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:21.366441 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:21.366597 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:21.366905 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:21.366909 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:21.366919 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:09:21.367084 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.367091 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.367892 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.367896 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.369059 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.369075 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.369649 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.369653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.369774 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.369850 (+ 76us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:09:21.370271 (+ 421us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:09:21.370279 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:09:21.370361 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 22:09:21.370459 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":130,"threads_started":1}
I20260811 22:09:21.379760 31455 raft_consensus.cc:1261] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Refusing update from remote peer 9f1cf3f6d31f427f8ce8030d53fde338: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:21.380198 31285 raft_consensus.cc:1261] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Refusing update from remote peer 9f1cf3f6d31f427f8ce8030d53fde338: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:21.381031 31549 consensus_queue.cc:1035] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:21.381896 31554 consensus_queue.cc:1035] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:21.417007 31540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.413182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:57777 (local address 127.28.47.129:40549)
0811 22:09:21.413315 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:21.413321 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:21.413345 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:09:21.413462 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:21.413465 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:21.413516 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:21.413609 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:21.413873 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.414512 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.415640 (+ 1128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.416024 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.416129 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.416459 (+ 330us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:21.416548 (+ 89us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:21.416656 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:21.416769 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 22:09:21.416859 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 22:09:21.417007 31519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:21.413043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:40549 (local address 127.28.47.131:57777)
0811 22:09:21.413253 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:21.413270 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:21.413370 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:21.413631 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:21.413634 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:21.413642 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:09:21.413747 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.413754 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.414677 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.414681 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.415493 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:21.415525 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:21.416135 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:21.416138 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:21.416226 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:21.416288 (+ 62us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:09:21.416662 (+ 374us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:09:21.416672 (+ 10us) client_negotiation.cc:769] Sending connection context
0811 22:09:21.416755 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:09:21.416848 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260811 22:09:21.450098 30935 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:21.450337 30935 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:21.453951 30935 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90: Sending DeleteTablet for 3 replicas of tablet 01d7b950e7bf4cf693530ce1d9122c05
I20260811 22:09:21.455338 31261 tablet_service.cc:1514] Processing DeleteTablet for tablet 01d7b950e7bf4cf693530ce1d9122c05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:21 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:53668
I20260811 22:09:21.455389 31094 tablet_service.cc:1514] Processing DeleteTablet for tablet 01d7b950e7bf4cf693530ce1d9122c05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:21 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:45318
I20260811 22:09:21.456143 31434 tablet_service.cc:1514] Processing DeleteTablet for tablet 01d7b950e7bf4cf693530ce1d9122c05 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:21 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:46814
I20260811 22:09:21.456652 31596 tablet_replica.cc:330] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5: stopping tablet replica
I20260811 22:09:21.457065 31596 raft_consensus.cc:2229] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:21.457361 31596 raft_consensus.cc:2258] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:21.457265 30935 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33518:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:09:21.457768 30935 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:21.461268 31596 ts_tablet_manager.cc:1905] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:21.464995 31596 ts_tablet_manager.cc:1918] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:21.465173 31596 log.cc:1196] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/01d7b950e7bf4cf693530ce1d9122c05
I20260811 22:09:21.465376 31597 tablet_replica.cc:330] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338: stopping tablet replica
I20260811 22:09:21.465668 31094 tablet_service.cc:1467] Processing CreateTablet for tablet e4df9dfa108241648a9b829107272e5f (DEFAULT_TABLE table=test-table [id=95c4677555fe49f1ab264251a7b53201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:21.465667 31434 tablet_service.cc:1467] Processing CreateTablet for tablet e4df9dfa108241648a9b829107272e5f (DEFAULT_TABLE table=test-table [id=95c4677555fe49f1ab264251a7b53201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:21.465808 31597 raft_consensus.cc:2229] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:09:21.465965 31094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4df9dfa108241648a9b829107272e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:21.466004 31434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4df9dfa108241648a9b829107272e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:21.466216 31597 raft_consensus.cc:2258] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:21.467911 31597 ts_tablet_manager.cc:1905] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:21.468549 31543 tablet_bootstrap.cc:492] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356: Bootstrap starting.
I20260811 22:09:21.469806 31261 tablet_service.cc:1467] Processing CreateTablet for tablet e4df9dfa108241648a9b829107272e5f (DEFAULT_TABLE table=test-table [id=95c4677555fe49f1ab264251a7b53201]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:21.470115 31261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e4df9dfa108241648a9b829107272e5f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:21.470487 31543 tablet_bootstrap.cc:654] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:21.470990 31597 ts_tablet_manager.cc:1918] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:21.471084 31597 log.cc:1196] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/wal/wals/01d7b950e7bf4cf693530ce1d9122c05
I20260811 22:09:21.471474 31597 ts_tablet_manager.cc:1939] T 01d7b950e7bf4cf693530ce1d9122c05 P 9f1cf3f6d31f427f8ce8030d53fde338: Deleting consensus metadata
I20260811 22:09:21.471537 31596 ts_tablet_manager.cc:1939] T 01d7b950e7bf4cf693530ce1d9122c05 P b5576f14aca34946aefc1f726b3ad5f5: Deleting consensus metadata
I20260811 22:09:21.471827 31598 tablet_replica.cc:330] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356: stopping tablet replica
I20260811 22:09:21.472165 31598 raft_consensus.cc:2229] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:21.472368 31598 raft_consensus.cc:2258] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:21.472751 31542 tablet_bootstrap.cc:492] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5: Bootstrap starting.
I20260811 22:09:21.473737 31542 tablet_bootstrap.cc:654] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:21.476495 31541 tablet_bootstrap.cc:492] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338: Bootstrap starting.
I20260811 22:09:21.477564 31541 tablet_bootstrap.cc:654] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:21.478909 30917 catalog_manager.cc:4841] TS 9f1cf3f6d31f427f8ce8030d53fde338 (127.28.47.129:40549): tablet 01d7b950e7bf4cf693530ce1d9122c05 (table test-table [id=49416707247b41e4a00703c35e888fb5]) successfully deleted
I20260811 22:09:21.481360 31542 tablet_bootstrap.cc:492] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5: No bootstrap required, opened a new log
I20260811 22:09:21.481467 31542 ts_tablet_manager.cc:1397] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5: Time spent bootstrapping tablet: real 0.009s user 0.000s sys 0.002s
I20260811 22:09:21.481986 31542 raft_consensus.cc:348] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.482111 31542 raft_consensus.cc:374] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:21.482141 31542 raft_consensus.cc:729] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5576f14aca34946aefc1f726b3ad5f5, State: Initialized, Role: FOLLOWER
I20260811 22:09:21.482270 31542 consensus_queue.cc:260] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.482515 31542 ts_tablet_manager.cc:1428] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:09:21.483299 31598 ts_tablet_manager.cc:1905] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:21.485316 31541 tablet_bootstrap.cc:492] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338: No bootstrap required, opened a new log
I20260811 22:09:21.485430 31541 ts_tablet_manager.cc:1397] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260811 22:09:21.485931 31541 raft_consensus.cc:348] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.486054 31541 raft_consensus.cc:374] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:21.486083 31541 raft_consensus.cc:729] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1cf3f6d31f427f8ce8030d53fde338, State: Initialized, Role: FOLLOWER
I20260811 22:09:21.486203 31541 consensus_queue.cc:260] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.486507 31541 ts_tablet_manager.cc:1428] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:09:21.487252 30916 catalog_manager.cc:4841] TS b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130:39123): tablet 01d7b950e7bf4cf693530ce1d9122c05 (table test-table [id=49416707247b41e4a00703c35e888fb5]) successfully deleted
I20260811 22:09:21.496966 31543 tablet_bootstrap.cc:492] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356: No bootstrap required, opened a new log
I20260811 22:09:21.497095 31543 ts_tablet_manager.cc:1397] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356: Time spent bootstrapping tablet: real 0.029s user 0.003s sys 0.000s
I20260811 22:09:21.497659 31543 raft_consensus.cc:348] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.497803 31543 raft_consensus.cc:374] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:21.497841 31543 raft_consensus.cc:729] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0de1775bd76b49fdb7eac8c179567356, State: Initialized, Role: FOLLOWER
I20260811 22:09:21.497979 31598 ts_tablet_manager.cc:1918] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:21.497979 31543 consensus_queue.cc:260] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.498229 31598 log.cc:1196] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/01d7b950e7bf4cf693530ce1d9122c05
I20260811 22:09:21.498378 31543 ts_tablet_manager.cc:1428] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:09:21.498723 31598 ts_tablet_manager.cc:1939] T 01d7b950e7bf4cf693530ce1d9122c05 P 0de1775bd76b49fdb7eac8c179567356: Deleting consensus metadata
I20260811 22:09:21.500150 30917 catalog_manager.cc:4841] TS 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131:39377): tablet 01d7b950e7bf4cf693530ce1d9122c05 (table test-table [id=49416707247b41e4a00703c35e888fb5]) successfully deleted
I20260811 22:09:21.615236 31592 raft_consensus.cc:482] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:21.615439 31592 raft_consensus.cc:504] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.616015 31592 leader_election.cc:288] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130:39123), 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131:39377)
I20260811 22:09:21.616503 31285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4df9dfa108241648a9b829107272e5f" candidate_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5576f14aca34946aefc1f726b3ad5f5" is_pre_election: true
I20260811 22:09:21.616611 31455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4df9dfa108241648a9b829107272e5f" candidate_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0de1775bd76b49fdb7eac8c179567356" is_pre_election: true
I20260811 22:09:21.616693 31285 raft_consensus.cc:2454] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1cf3f6d31f427f8ce8030d53fde338 in term 0.
I20260811 22:09:21.616796 31455 raft_consensus.cc:2454] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1cf3f6d31f427f8ce8030d53fde338 in term 0.
I20260811 22:09:21.617057 31039 leader_election.cc:302] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1cf3f6d31f427f8ce8030d53fde338, b5576f14aca34946aefc1f726b3ad5f5; no voters:
I20260811 22:09:21.617350 31592 raft_consensus.cc:2781] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:21.617431 31592 raft_consensus.cc:482] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:21.617486 31592 raft_consensus.cc:3037] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:21.618549 31592 raft_consensus.cc:504] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.619004 31592 leader_election.cc:288] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [CANDIDATE]: Term 1 election: Requested vote from peers b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130:39123), 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131:39377)
I20260811 22:09:21.619504 31285 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4df9dfa108241648a9b829107272e5f" candidate_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5576f14aca34946aefc1f726b3ad5f5"
I20260811 22:09:21.619688 31285 raft_consensus.cc:3037] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:21.619684 31455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e4df9dfa108241648a9b829107272e5f" candidate_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0de1775bd76b49fdb7eac8c179567356"
I20260811 22:09:21.619832 31455 raft_consensus.cc:3037] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:21.621193 31285 raft_consensus.cc:2454] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1cf3f6d31f427f8ce8030d53fde338 in term 1.
I20260811 22:09:21.621232 31455 raft_consensus.cc:2454] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1cf3f6d31f427f8ce8030d53fde338 in term 1.
I20260811 22:09:21.621663 31039 leader_election.cc:302] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1cf3f6d31f427f8ce8030d53fde338, b5576f14aca34946aefc1f726b3ad5f5; no voters:
I20260811 22:09:21.622079 31592 raft_consensus.cc:2781] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:21.622241 31592 raft_consensus.cc:686] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1cf3f6d31f427f8ce8030d53fde338, State: Running, Role: LEADER
I20260811 22:09:21.622427 31592 consensus_queue.cc:237] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:21.624548 30935 catalog_manager.cc:5495] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1cf3f6d31f427f8ce8030d53fde338 (127.28.47.129). New cstate: current_term: 1 leader_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } health_report { overall_health: UNKNOWN } } }
I20260811 22:09:21.654109 31455 raft_consensus.cc:1261] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Refusing update from remote peer 9f1cf3f6d31f427f8ce8030d53fde338: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:21.654573 31285 raft_consensus.cc:1261] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Refusing update from remote peer 9f1cf3f6d31f427f8ce8030d53fde338: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:21.654575 31592 consensus_queue.cc:1035] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:21.655085 31592 consensus_queue.cc:1035] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:09:21.672606 31575 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:09:23.243464 30933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:23.243639 30933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33518:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:23.246846 30933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90: Sending DeleteTablet for 3 replicas of tablet e4df9dfa108241648a9b829107272e5f
I20260811 22:09:23.247645 31094 tablet_service.cc:1514] Processing DeleteTablet for tablet e4df9dfa108241648a9b829107272e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:23 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:45318
I20260811 22:09:23.247735 31261 tablet_service.cc:1514] Processing DeleteTablet for tablet e4df9dfa108241648a9b829107272e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:23 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:53668
I20260811 22:09:23.247984 31434 tablet_service.cc:1514] Processing DeleteTablet for tablet e4df9dfa108241648a9b829107272e5f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:23 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:46814
I20260811 22:09:23.248090 31631 tablet_replica.cc:330] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338: stopping tablet replica
I20260811 22:09:23.248248 31631 raft_consensus.cc:2229] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:09:23.248262 31629 tablet_replica.cc:330] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5: stopping tablet replica
I20260811 22:09:23.248492 31629 raft_consensus.cc:2229] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:23.248678 31629 raft_consensus.cc:2258] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:23.248709 31631 raft_consensus.cc:2258] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:23.249387 31632 tablet_replica.cc:330] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356: stopping tablet replica
I20260811 22:09:23.249503 31631 ts_tablet_manager.cc:1905] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:23.249601 31632 raft_consensus.cc:2229] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:23.249620 31629 ts_tablet_manager.cc:1905] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:23.249783 31632 raft_consensus.cc:2258] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:23.250757 31632 ts_tablet_manager.cc:1905] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:23.252671 31629 ts_tablet_manager.cc:1918] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:09:23.252708 31631 ts_tablet_manager.cc:1918] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:09:23.252784 31631 log.cc:1196] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/wal/wals/e4df9dfa108241648a9b829107272e5f
I20260811 22:09:23.252890 31629 log.cc:1196] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/e4df9dfa108241648a9b829107272e5f
I20260811 22:09:23.253134 31631 ts_tablet_manager.cc:1939] T e4df9dfa108241648a9b829107272e5f P 9f1cf3f6d31f427f8ce8030d53fde338: Deleting consensus metadata
I20260811 22:09:23.253304 31629 ts_tablet_manager.cc:1939] T e4df9dfa108241648a9b829107272e5f P b5576f14aca34946aefc1f726b3ad5f5: Deleting consensus metadata
I20260811 22:09:23.254153 30916 catalog_manager.cc:4841] TS b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130:39123): tablet e4df9dfa108241648a9b829107272e5f (table test-table [id=95c4677555fe49f1ab264251a7b53201]) successfully deleted
I20260811 22:09:23.254179 30917 catalog_manager.cc:4841] TS 9f1cf3f6d31f427f8ce8030d53fde338 (127.28.47.129:40549): tablet e4df9dfa108241648a9b829107272e5f (table test-table [id=95c4677555fe49f1ab264251a7b53201]) successfully deleted
I20260811 22:09:23.255095 31632 ts_tablet_manager.cc:1918] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:09:23.255226 31632 log.cc:1196] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/e4df9dfa108241648a9b829107272e5f
I20260811 22:09:23.255544 31632 ts_tablet_manager.cc:1939] T e4df9dfa108241648a9b829107272e5f P 0de1775bd76b49fdb7eac8c179567356: Deleting consensus metadata
I20260811 22:09:23.256664 30917 catalog_manager.cc:4841] TS 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131:39377): tablet e4df9dfa108241648a9b829107272e5f (table test-table [id=95c4677555fe49f1ab264251a7b53201]) successfully deleted
Aug 11 22:09:23 dist-test-slave-vxj2 krb5kdc[30880](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486163, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:09:23 dist-test-slave-vxj2 krb5kdc[30880](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486163, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:23.286633 31641 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:23.271641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33542 (local address 127.28.47.190:44257)
0811 22:09:23.272027 (+ 386us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:23.272038 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:23.272056 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:09:23.272248 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:23.272251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:23.272302 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:23.272413 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:23.273227 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.273794 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.274714 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.274963 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.275079 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:23.282726 (+ 7647us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:23.282737 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:23.282739 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:23.282779 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:23.284443 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:23.285010 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:23.285014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:23.285016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:23.285120 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:23.285520 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:23.285525 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:23.285526 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:23.285827 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:23.286008 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:23.286261 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 22:09:23.286418 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":295,"thread_start_us":162,"threads_started":1}
I20260811 22:09:23.290222 30933 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:09:23.290827 30933 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:23.297314 31261 tablet_service.cc:1467] Processing CreateTablet for tablet 5eac522de9954ec4b59c81028fd7b497 (DEFAULT_TABLE table=test-table [id=aea25997b6124966acb0603b2a7ace15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:23.297369 31094 tablet_service.cc:1467] Processing CreateTablet for tablet 5eac522de9954ec4b59c81028fd7b497 (DEFAULT_TABLE table=test-table [id=aea25997b6124966acb0603b2a7ace15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:23.297735 31261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eac522de9954ec4b59c81028fd7b497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:23.297750 31094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eac522de9954ec4b59c81028fd7b497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:23.297770 31434 tablet_service.cc:1467] Processing CreateTablet for tablet 5eac522de9954ec4b59c81028fd7b497 (DEFAULT_TABLE table=test-table [id=aea25997b6124966acb0603b2a7ace15]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:23.298120 31434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5eac522de9954ec4b59c81028fd7b497. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:23.300724 31643 tablet_bootstrap.cc:492] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356: Bootstrap starting.
I20260811 22:09:23.300937 31644 tablet_bootstrap.cc:492] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338: Bootstrap starting.
I20260811 22:09:23.301204 31642 tablet_bootstrap.cc:492] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5: Bootstrap starting.
I20260811 22:09:23.301770 31643 tablet_bootstrap.cc:654] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:23.301985 31644 tablet_bootstrap.cc:654] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:23.302505 31642 tablet_bootstrap.cc:654] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:23.303243 31643 tablet_bootstrap.cc:492] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356: No bootstrap required, opened a new log
I20260811 22:09:23.303354 31643 ts_tablet_manager.cc:1397] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:23.303514 31644 tablet_bootstrap.cc:492] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338: No bootstrap required, opened a new log
I20260811 22:09:23.303663 31644 ts_tablet_manager.cc:1397] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:23.303911 31643 raft_consensus.cc:348] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.304013 31642 tablet_bootstrap.cc:492] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5: No bootstrap required, opened a new log
I20260811 22:09:23.304050 31643 raft_consensus.cc:374] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:23.304090 31643 raft_consensus.cc:729] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0de1775bd76b49fdb7eac8c179567356, State: Initialized, Role: FOLLOWER
I20260811 22:09:23.304100 31642 ts_tablet_manager.cc:1397] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:09:23.304246 31643 consensus_queue.cc:260] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.304354 31644 raft_consensus.cc:348] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.304488 31644 raft_consensus.cc:374] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:23.304522 31644 raft_consensus.cc:729] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1cf3f6d31f427f8ce8030d53fde338, State: Initialized, Role: FOLLOWER
I20260811 22:09:23.304546 31643 ts_tablet_manager.cc:1428] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:09:23.304569 31642 raft_consensus.cc:348] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.304673 31642 raft_consensus.cc:374] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:23.304708 31642 raft_consensus.cc:729] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5576f14aca34946aefc1f726b3ad5f5, State: Initialized, Role: FOLLOWER
I20260811 22:09:23.304668 31644 consensus_queue.cc:260] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.304885 31642 consensus_queue.cc:260] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.304952 31644 ts_tablet_manager.cc:1428] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:09:23.305315 31642 ts_tablet_manager.cc:1428] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 22:09:23.309710 31649 raft_consensus.cc:482] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:23.309904 31649 raft_consensus.cc:504] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.311194 31649 leader_election.cc:288] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9f1cf3f6d31f427f8ce8030d53fde338 (127.28.47.129:40549), 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131:39377)
I20260811 22:09:23.316993 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:23.311608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:34155 (local address 127.28.47.129:40549)
0811 22:09:23.311939 (+ 331us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:23.311953 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:23.312076 (+ 123us) server_negotiation.cc:407] Connection header received
0811 22:09:23.312211 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:23.312214 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:23.312274 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:23.312391 (+ 117us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:23.312793 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.313630 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.315592 (+ 1962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.316300 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.316483 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:23.316542 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:23.316742 (+ 200us) server_negotiation.cc:299] Negotiation successful
0811 22:09:23.316835 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":217,"thread_start_us":100,"threads_started":1}
I20260811 22:09:23.317014 31652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:23.311545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:40549 (local address 127.28.47.130:34155)
0811 22:09:23.311946 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:23.311981 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:23.312106 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:23.312421 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:23.312424 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:23.312443 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:23.312607 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:23.312615 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.313783 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.313786 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:23.315405 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:23.315419 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.316466 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.316471 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:23.316583 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:23.316614 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:09:23.316713 (+ 99us) client_negotiation.cc:240] Negotiation successful
0811 22:09:23.316817 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"mutex_wait_us":31,"thread_start_us":170,"threads_started":1}
I20260811 22:09:23.317437 31654 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:23.312018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:39377 (local address 127.28.47.130:39219)
0811 22:09:23.312651 (+ 633us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:23.312666 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:23.312761 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:23.313046 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:23.313048 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:23.313062 (+ 14us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:23.313210 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:23.313217 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.314130 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.314134 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:23.315540 (+ 1406us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:23.315551 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.316639 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.316642 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:23.316766 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:23.316790 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:09:23.316874 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 22:09:23.316945 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":566,"mutex_wait_us":1,"thread_start_us":103,"threads_started":1}
I20260811 22:09:23.317615 31655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:23.312148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:39219 (local address 127.28.47.131:39377)
0811 22:09:23.312452 (+ 304us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:23.312459 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:23.312772 (+ 313us) server_negotiation.cc:407] Connection header received
0811 22:09:23.312887 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:23.312891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:23.312939 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:23.313035 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:23.313327 (+ 292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.313987 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.315715 (+ 1728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.316362 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.316483 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:23.316542 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:23.317081 (+ 539us) server_negotiation.cc:299] Negotiation successful
0811 22:09:23.317452 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":147,"threads_started":1}
I20260811 22:09:23.317687 31120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eac522de9954ec4b59c81028fd7b497" candidate_uuid: "b5576f14aca34946aefc1f726b3ad5f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" is_pre_election: true
I20260811 22:09:23.317901 31120 raft_consensus.cc:2454] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5576f14aca34946aefc1f726b3ad5f5 in term 0.
I20260811 22:09:23.318423 31202 leader_election.cc:302] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1cf3f6d31f427f8ce8030d53fde338, b5576f14aca34946aefc1f726b3ad5f5; no voters:
I20260811 22:09:23.318535 31455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eac522de9954ec4b59c81028fd7b497" candidate_uuid: "b5576f14aca34946aefc1f726b3ad5f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0de1775bd76b49fdb7eac8c179567356" is_pre_election: true
I20260811 22:09:23.318701 31455 raft_consensus.cc:2454] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b5576f14aca34946aefc1f726b3ad5f5 in term 0.
I20260811 22:09:23.318740 31649 raft_consensus.cc:2781] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:23.318806 31649 raft_consensus.cc:482] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:23.318856 31649 raft_consensus.cc:3037] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:23.319950 31649 raft_consensus.cc:504] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.320295 31649 leader_election.cc:288] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [CANDIDATE]: Term 1 election: Requested vote from peers 9f1cf3f6d31f427f8ce8030d53fde338 (127.28.47.129:40549), 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131:39377)
I20260811 22:09:23.320730 31120 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eac522de9954ec4b59c81028fd7b497" candidate_uuid: "b5576f14aca34946aefc1f726b3ad5f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9f1cf3f6d31f427f8ce8030d53fde338"
I20260811 22:09:23.320768 31455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5eac522de9954ec4b59c81028fd7b497" candidate_uuid: "b5576f14aca34946aefc1f726b3ad5f5" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0de1775bd76b49fdb7eac8c179567356"
I20260811 22:09:23.320881 31120 raft_consensus.cc:3037] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:23.320936 31455 raft_consensus.cc:3037] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:23.322168 31455 raft_consensus.cc:2454] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5576f14aca34946aefc1f726b3ad5f5 in term 1.
I20260811 22:09:23.322184 31120 raft_consensus.cc:2454] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b5576f14aca34946aefc1f726b3ad5f5 in term 1.
I20260811 22:09:23.322520 31202 leader_election.cc:302] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9f1cf3f6d31f427f8ce8030d53fde338, b5576f14aca34946aefc1f726b3ad5f5; no voters:
I20260811 22:09:23.322716 31649 raft_consensus.cc:2781] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:23.322989 31649 raft_consensus.cc:686] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 LEADER]: Becoming Leader. State: Replica: b5576f14aca34946aefc1f726b3ad5f5, State: Running, Role: LEADER
I20260811 22:09:23.323334 31649 consensus_queue.cc:237] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.326061 30931 catalog_manager.cc:5495] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 reported cstate change: term changed from 0 to 1, leader changed from <none> to b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130). New cstate: current_term: 1 leader_uuid: "b5576f14aca34946aefc1f726b3ad5f5" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } health_report { overall_health: UNKNOWN } } }
I20260811 22:09:23.346202 31659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:23.341378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53670 (local address 127.28.47.130:39123)
0811 22:09:23.341702 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:23.341709 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:23.341723 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:09:23.341803 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:23.341805 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:23.341880 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:23.341996 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:23.342231 (+ 235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.342857 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.344063 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.344335 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.344436 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:23.344941 (+ 505us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:23.345627 (+ 686us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:23.345757 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:23.345870 (+ 113us) server_negotiation.cc:299] Negotiation successful
0811 22:09:23.345939 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":151,"threads_started":1}
I20260811 22:09:23.359288 31655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:23.355599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:59887 (local address 127.28.47.131:39377)
0811 22:09:23.355855 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:23.355858 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:23.356026 (+ 168us) server_negotiation.cc:407] Connection header received
0811 22:09:23.356245 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:23.356248 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:23.356298 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:23.356367 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:23.356641 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.357195 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.358060 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.358304 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.358397 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:23.358808 (+ 411us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:23.358906 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:23.358976 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:23.359115 (+ 139us) server_negotiation.cc:299] Negotiation successful
0811 22:09:23.359174 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":164}
I20260811 22:09:23.359305 31662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:23.355486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:39377 (local address 127.28.47.130:59887)
0811 22:09:23.355886 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:23.355912 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:23.356113 (+ 201us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:23.356399 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:23.356401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:23.356408 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:09:23.356531 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:23.356536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.357297 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.357299 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:23.357931 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:23.357941 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.358438 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.358441 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:23.358525 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:23.358582 (+ 57us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:09:23.359014 (+ 432us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:09:23.359022 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:09:23.359083 (+ 61us) client_negotiation.cc:240] Negotiation successful
0811 22:09:23.359165 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":134,"threads_started":1}
I20260811 22:09:23.366398 31120 raft_consensus.cc:1261] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Refusing update from remote peer b5576f14aca34946aefc1f726b3ad5f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:23.366389 31455 raft_consensus.cc:1261] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Refusing update from remote peer b5576f14aca34946aefc1f726b3ad5f5: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:23.366912 31649 consensus_queue.cc:1035] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:23.367117 31656 consensus_queue.cc:1035] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:23.389261 31659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:23.384876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:46959 (local address 127.28.47.130:39123)
0811 22:09:23.384992 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:23.384998 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:23.385333 (+ 335us) server_negotiation.cc:407] Connection header received
0811 22:09:23.385520 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:23.385523 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:23.385569 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:23.385613 (+ 44us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:23.386001 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.386696 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.387957 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.388248 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.388328 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:23.388685 (+ 357us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:23.388782 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:23.388828 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:23.389012 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 22:09:23.389071 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 22:09:23.389861 31670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:23.384496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:39123 (local address 127.28.47.131:46959)
0811 22:09:23.385237 (+ 741us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:23.385252 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:23.385407 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:23.385775 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:23.385778 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:23.385786 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:09:23.385888 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:23.385894 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.386993 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.386996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:23.387807 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:23.387819 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.388445 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.388448 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:23.388523 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:23.388552 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:09:23.388898 (+ 346us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:09:23.388906 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:09:23.388993 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:09:23.389178 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":635,"thread_start_us":106,"threads_started":1}
I20260811 22:09:23.421803 30933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:23.422034 30933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:23.424639 30933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90: Sending DeleteTablet for 3 replicas of tablet 5eac522de9954ec4b59c81028fd7b497
I20260811 22:09:23.425468 31434 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eac522de9954ec4b59c81028fd7b497 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:23 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:46814
I20260811 22:09:23.425634 31261 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eac522de9954ec4b59c81028fd7b497 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:23 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:53668
I20260811 22:09:23.425688 31632 tablet_replica.cc:330] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356: stopping tablet replica
I20260811 22:09:23.425856 31629 tablet_replica.cc:330] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5: stopping tablet replica
I20260811 22:09:23.425889 31632 raft_consensus.cc:2229] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:23.425994 31629 raft_consensus.cc:2229] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:09:23.426054 31632 raft_consensus.cc:2258] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:23.426383 31629 raft_consensus.cc:2258] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:23.426465 31094 tablet_service.cc:1514] Processing DeleteTablet for tablet 5eac522de9954ec4b59c81028fd7b497 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:23 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:45318
I20260811 22:09:23.426676 31631 tablet_replica.cc:330] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338: stopping tablet replica
I20260811 22:09:23.426612 30931 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:09:23.426790 31631 raft_consensus.cc:2229] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:23.426923 31631 raft_consensus.cc:2258] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 22:09:23.427095 30931 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:23.427361 31629 ts_tablet_manager.cc:1905] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:23.427835 31632 ts_tablet_manager.cc:1905] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:23.430644 31631 ts_tablet_manager.cc:1905] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:23.431115 31632 ts_tablet_manager.cc:1918] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:23.431193 31632 log.cc:1196] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/5eac522de9954ec4b59c81028fd7b497
I20260811 22:09:23.431244 31629 ts_tablet_manager.cc:1918] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:23.431336 31629 log.cc:1196] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/5eac522de9954ec4b59c81028fd7b497
I20260811 22:09:23.431474 31632 ts_tablet_manager.cc:1939] T 5eac522de9954ec4b59c81028fd7b497 P 0de1775bd76b49fdb7eac8c179567356: Deleting consensus metadata
I20260811 22:09:23.431677 31629 ts_tablet_manager.cc:1939] T 5eac522de9954ec4b59c81028fd7b497 P b5576f14aca34946aefc1f726b3ad5f5: Deleting consensus metadata
I20260811 22:09:23.432337 30917 catalog_manager.cc:4841] TS 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131:39377): tablet 5eac522de9954ec4b59c81028fd7b497 (table test-table [id=aea25997b6124966acb0603b2a7ace15]) successfully deleted
I20260811 22:09:23.432554 30916 catalog_manager.cc:4841] TS b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130:39123): tablet 5eac522de9954ec4b59c81028fd7b497 (table test-table [id=aea25997b6124966acb0603b2a7ace15]) successfully deleted
I20260811 22:09:23.433676 31631 ts_tablet_manager.cc:1918] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:23.433784 31631 log.cc:1196] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-0/wal/wals/5eac522de9954ec4b59c81028fd7b497
I20260811 22:09:23.434103 31631 ts_tablet_manager.cc:1939] T 5eac522de9954ec4b59c81028fd7b497 P 9f1cf3f6d31f427f8ce8030d53fde338: Deleting consensus metadata
I20260811 22:09:23.434839 30917 catalog_manager.cc:4841] TS 9f1cf3f6d31f427f8ce8030d53fde338 (127.28.47.129:40549): tablet 5eac522de9954ec4b59c81028fd7b497 (table test-table [id=aea25997b6124966acb0603b2a7ace15]) successfully deleted
I20260811 22:09:23.437410 31094 tablet_service.cc:1467] Processing CreateTablet for tablet 994bf34cc0ae4138a718d8878c842855 (DEFAULT_TABLE table=test-table [id=506ec464a09a4b3a9b4436c691906f2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:23.437606 31261 tablet_service.cc:1467] Processing CreateTablet for tablet 994bf34cc0ae4138a718d8878c842855 (DEFAULT_TABLE table=test-table [id=506ec464a09a4b3a9b4436c691906f2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:23.437736 31094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 994bf34cc0ae4138a718d8878c842855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:23.437954 31261 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 994bf34cc0ae4138a718d8878c842855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:23.438831 31434 tablet_service.cc:1467] Processing CreateTablet for tablet 994bf34cc0ae4138a718d8878c842855 (DEFAULT_TABLE table=test-table [id=506ec464a09a4b3a9b4436c691906f2c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:23.439133 31434 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 994bf34cc0ae4138a718d8878c842855. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:23.440574 31644 tablet_bootstrap.cc:492] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338: Bootstrap starting.
I20260811 22:09:23.440582 31642 tablet_bootstrap.cc:492] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5: Bootstrap starting.
I20260811 22:09:23.441533 31644 tablet_bootstrap.cc:654] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:23.442613 31642 tablet_bootstrap.cc:654] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:23.443096 31644 tablet_bootstrap.cc:492] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338: No bootstrap required, opened a new log
I20260811 22:09:23.443194 31644 ts_tablet_manager.cc:1397] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:09:23.443728 31644 raft_consensus.cc:348] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.443866 31644 raft_consensus.cc:374] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:23.443907 31644 raft_consensus.cc:729] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9f1cf3f6d31f427f8ce8030d53fde338, State: Initialized, Role: FOLLOWER
I20260811 22:09:23.443954 31643 tablet_bootstrap.cc:492] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356: Bootstrap starting.
I20260811 22:09:23.444036 31644 consensus_queue.cc:260] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.444347 31644 ts_tablet_manager.cc:1428] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260811 22:09:23.445493 31643 tablet_bootstrap.cc:654] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:23.447145 31643 tablet_bootstrap.cc:492] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356: No bootstrap required, opened a new log
I20260811 22:09:23.447273 31643 ts_tablet_manager.cc:1397] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:23.447779 31643 raft_consensus.cc:348] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.447937 31643 raft_consensus.cc:374] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:23.447943 31642 tablet_bootstrap.cc:492] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5: No bootstrap required, opened a new log
I20260811 22:09:23.447986 31643 raft_consensus.cc:729] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0de1775bd76b49fdb7eac8c179567356, State: Initialized, Role: FOLLOWER
I20260811 22:09:23.448053 31642 ts_tablet_manager.cc:1397] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.001s
I20260811 22:09:23.448163 31643 consensus_queue.cc:260] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.449000 31643 ts_tablet_manager.cc:1428] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:09:23.448956 31642 raft_consensus.cc:348] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.449077 31642 raft_consensus.cc:374] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:23.449121 31642 raft_consensus.cc:729] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b5576f14aca34946aefc1f726b3ad5f5, State: Initialized, Role: FOLLOWER
I20260811 22:09:23.449322 31642 consensus_queue.cc:260] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.450260 31642 ts_tablet_manager.cc:1428] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:09:23.502103 31554 raft_consensus.cc:482] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:23.502321 31554 raft_consensus.cc:504] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.502934 31554 leader_election.cc:288] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130:39123), 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131:39377)
I20260811 22:09:23.503446 31455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "994bf34cc0ae4138a718d8878c842855" candidate_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0de1775bd76b49fdb7eac8c179567356" is_pre_election: true
I20260811 22:09:23.503477 31284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "994bf34cc0ae4138a718d8878c842855" candidate_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5576f14aca34946aefc1f726b3ad5f5" is_pre_election: true
I20260811 22:09:23.503634 31455 raft_consensus.cc:2454] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1cf3f6d31f427f8ce8030d53fde338 in term 0.
I20260811 22:09:23.503654 31284 raft_consensus.cc:2454] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9f1cf3f6d31f427f8ce8030d53fde338 in term 0.
I20260811 22:09:23.504001 31040 leader_election.cc:302] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0de1775bd76b49fdb7eac8c179567356, 9f1cf3f6d31f427f8ce8030d53fde338; no voters:
I20260811 22:09:23.504261 31554 raft_consensus.cc:2781] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:23.504339 31554 raft_consensus.cc:482] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:23.504397 31554 raft_consensus.cc:3037] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:23.505614 31554 raft_consensus.cc:504] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.506050 31554 leader_election.cc:288] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [CANDIDATE]: Term 1 election: Requested vote from peers b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130:39123), 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131:39377)
I20260811 22:09:23.506488 31284 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "994bf34cc0ae4138a718d8878c842855" candidate_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b5576f14aca34946aefc1f726b3ad5f5"
I20260811 22:09:23.506500 31455 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "994bf34cc0ae4138a718d8878c842855" candidate_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0de1775bd76b49fdb7eac8c179567356"
I20260811 22:09:23.506642 31455 raft_consensus.cc:3037] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:23.506628 31284 raft_consensus.cc:3037] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:23.507908 31455 raft_consensus.cc:2454] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1cf3f6d31f427f8ce8030d53fde338 in term 1.
I20260811 22:09:23.508306 31040 leader_election.cc:302] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0de1775bd76b49fdb7eac8c179567356, 9f1cf3f6d31f427f8ce8030d53fde338; no voters:
I20260811 22:09:23.508509 31554 raft_consensus.cc:2781] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:23.508585 31284 raft_consensus.cc:2454] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9f1cf3f6d31f427f8ce8030d53fde338 in term 1.
I20260811 22:09:23.508658 31554 raft_consensus.cc:686] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [term 1 LEADER]: Becoming Leader. State: Replica: 9f1cf3f6d31f427f8ce8030d53fde338, State: Running, Role: LEADER
I20260811 22:09:23.508837 31554 consensus_queue.cc:237] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } }
I20260811 22:09:23.511113 30931 catalog_manager.cc:5495] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 reported cstate change: term changed from 0 to 1, leader changed from <none> to 9f1cf3f6d31f427f8ce8030d53fde338 (127.28.47.129). New cstate: current_term: 1 leader_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9f1cf3f6d31f427f8ce8030d53fde338" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40549 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 } health_report { overall_health: UNKNOWN } } }
I20260811 22:09:23.536872 31653 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:23.533224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45332 (local address 127.28.47.129:40549)
0811 22:09:23.533358 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:23.533361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:23.533448 (+ 87us) server_negotiation.cc:407] Connection header received
0811 22:09:23.533594 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:23.533596 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:23.533639 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:23.533694 (+ 55us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:23.533998 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.534524 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.535709 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:23.535928 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:23.536008 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:23.536337 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:23.536421 (+ 84us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:23.536498 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:23.536648 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 22:09:23.536752 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:09:23.543784 31455 raft_consensus.cc:1261] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Refusing update from remote peer 9f1cf3f6d31f427f8ce8030d53fde338: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:23.543963 31284 raft_consensus.cc:1261] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Refusing update from remote peer 9f1cf3f6d31f427f8ce8030d53fde338: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:23.544265 31622 consensus_queue.cc:1035] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0de1775bd76b49fdb7eac8c179567356" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39377 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:23.544422 31554 consensus_queue.cc:1035] T 994bf34cc0ae4138a718d8878c842855 P 9f1cf3f6d31f427f8ce8030d53fde338 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b5576f14aca34946aefc1f726b3ad5f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39123 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:09:23.562350 31612 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260811 22:09:25.139392 30933 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:25.139557 30933 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:25.142398 30933 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c122a810b719438eb6cce2cd66a2cc90: Sending DeleteTablet for 3 replicas of tablet 994bf34cc0ae4138a718d8878c842855
I20260811 22:09:25.143167 31094 tablet_service.cc:1514] Processing DeleteTablet for tablet 994bf34cc0ae4138a718d8878c842855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:25 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:45318
I20260811 22:09:25.143237 31261 tablet_service.cc:1514] Processing DeleteTablet for tablet 994bf34cc0ae4138a718d8878c842855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:25 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:53668
I20260811 22:09:25.143275 31434 tablet_service.cc:1514] Processing DeleteTablet for tablet 994bf34cc0ae4138a718d8878c842855 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:25 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:46814
I20260811 22:09:25.143743 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 31019
I20260811 22:09:25.150511 31705 tablet_replica.cc:330] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5: stopping tablet replica
I20260811 22:09:25.150791 31705 raft_consensus.cc:2229] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:25.151043 31705 raft_consensus.cc:2258] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:25.152477 31705 ts_tablet_manager.cc:1905] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:25.153052 31706 tablet_replica.cc:330] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356: stopping tablet replica
I20260811 22:09:25.153247 31706 raft_consensus.cc:2229] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:25.153404 31706 raft_consensus.cc:2258] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:25.154842 31706 ts_tablet_manager.cc:1905] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:25.156462 31705 ts_tablet_manager.cc:1918] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:09:25.156559 31705 log.cc:1196] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/994bf34cc0ae4138a718d8878c842855
I20260811 22:09:25.156852 31705 ts_tablet_manager.cc:1939] T 994bf34cc0ae4138a718d8878c842855 P b5576f14aca34946aefc1f726b3ad5f5: Deleting consensus metadata
I20260811 22:09:25.157337 31706 ts_tablet_manager.cc:1918] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260811 22:09:25.157428 31706 log.cc:1196] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/994bf34cc0ae4138a718d8878c842855
I20260811 22:09:25.157677 31706 ts_tablet_manager.cc:1939] T 994bf34cc0ae4138a718d8878c842855 P 0de1775bd76b49fdb7eac8c179567356: Deleting consensus metadata
I20260811 22:09:25.157975 30916 catalog_manager.cc:4841] TS b5576f14aca34946aefc1f726b3ad5f5 (127.28.47.130:39123): tablet 994bf34cc0ae4138a718d8878c842855 (table test-table [id=506ec464a09a4b3a9b4436c691906f2c]) successfully deleted
I20260811 22:09:25.158484 30917 catalog_manager.cc:4841] TS 0de1775bd76b49fdb7eac8c179567356 (127.28.47.131:39377): tablet 994bf34cc0ae4138a718d8878c842855 (table test-table [id=506ec464a09a4b3a9b4436c691906f2c]) successfully deleted
W20260811 22:09:25.162009 30917 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:40549 (error 108)
I20260811 22:09:25.162242 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 31183
I20260811 22:09:25.165536 31708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:25.165000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:40549 (local address 127.0.0.1:48326)
0811 22:09:25.165363 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:25.165440 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:40549: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":99,"threads_started":1}
W20260811 22:09:25.165838 30917 catalog_manager.cc:4584] TS 9f1cf3f6d31f427f8ce8030d53fde338 (127.28.47.129:40549): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 994bf34cc0ae4138a718d8878c842855: Network error: Client connection negotiation failed: client connection to 127.28.47.129:40549: connect: Connection refused (error 111)
I20260811 22:09:25.172828 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 31353
I20260811 22:09:25.185297 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 30897
2026-08-11T22:09:25Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (6597 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:25 dist-test-slave-vxj2 krb5kdc[31714](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:25 dist-test-slave-vxj2 krb5kdc[31714](info): set up 2 sockets
Aug 11 22:09:25 dist-test-slave-vxj2 krb5kdc[31714](info): commencing operation
krb5kdc: starting...
W20260811 22:09:26.372028 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:09:26 dist-test-slave-vxj2 krb5kdc[31714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:09:26.513453 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:35917
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:35917
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:26.640700 31731 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:26.641029 31731 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:26.641076 31731 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:26.645686 31731 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:26.645785 31731 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:26.645807 31731 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:26.645825 31731 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:26.645841 31731 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:26.652217 31731 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:35917
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:35917
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31731
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:26.653611 31731 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:26.655092 31731 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:09:26.666399 31731 server_base.cc:1034] running on GCE node
W20260811 22:09:26.666486 31741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:26.666383 31736 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:26.666383 31739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:26.667421 31731 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:26.668892 31731 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:26.670112 31731 hybrid_clock.cc:648] HybridClock initialized: now 1786486166670074 us; error 61 us; skew 500 ppm
Aug 11 22:09:26 dist-test-slave-vxj2 krb5kdc[31714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486166, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:26.673755 31731 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:09:26.674971 31731 webserver.cc:460] Webserver started at http://127.28.47.190:34503/ using document root <none> and password file <none>
I20260811 22:09:26.675593 31731 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:26.675683 31731 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:26.675954 31731 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:26.678442 31731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "a670e1d4cd834325adb9f2a5815f8b13"
format_stamp: "Formatted at 2026-08-11 22:09:26 on dist-test-slave-vxj2"
server_key: "8b0f5adbffc16ed92217dde364099e84"
server_key_iv: "130b92343df3015cb11bc1ff0409b22e"
server_key_version: "encryptionkey@0"
I20260811 22:09:26.679149 31731 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "a670e1d4cd834325adb9f2a5815f8b13"
format_stamp: "Formatted at 2026-08-11 22:09:26 on dist-test-slave-vxj2"
server_key: "8b0f5adbffc16ed92217dde364099e84"
server_key_iv: "130b92343df3015cb11bc1ff0409b22e"
server_key_version: "encryptionkey@0"
I20260811 22:09:26.685352 31731 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.001s
I20260811 22:09:26.688598 31751 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:26.690346 31731 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:09:26.690524 31731 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "a670e1d4cd834325adb9f2a5815f8b13"
format_stamp: "Formatted at 2026-08-11 22:09:26 on dist-test-slave-vxj2"
server_key: "8b0f5adbffc16ed92217dde364099e84"
server_key_iv: "130b92343df3015cb11bc1ff0409b22e"
server_key_version: "encryptionkey@0"
I20260811 22:09:26.690653 31731 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:26.716699 31731 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:26.721489 31731 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:26.721699 31731 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:26.730091 31731 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:35917
I20260811 22:09:26.730139 31810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:35917 every 8 connection(s)
I20260811 22:09:26.731191 31731 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:26.732329 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 31731
I20260811 22:09:26.732422 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:26.732712 28862 external_mini_cluster.cc:1428] Setting key a12570f1d5eb44f3083df7c94e23b4ae
I20260811 22:09:26.734683 31811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:26.741889 31811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13: Bootstrap starting.
Aug 11 22:09:26 dist-test-slave-vxj2 krb5kdc[31714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486166, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:26.744493 31811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:26.745327 31811 log.cc:824] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:26.747920 31811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13: No bootstrap required, opened a new log
I20260811 22:09:26.748791 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:26.734397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39832 (local address 127.28.47.190:35917)
0811 22:09:26.734850 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:26.734877 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:26.734919 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:09:26.735591 (+ 672us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:26.735609 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:26.735976 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:26.736277 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:26.737348 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:26.738435 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:26.739687 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:26.740204 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:26.740355 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:26.743390 (+ 3035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:26.743396 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:26.743411 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:26.743446 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:26.745422 (+ 1976us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:26.745997 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:26.746006 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:26.746015 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:26.746169 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:26.746573 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:26.746576 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:26.746577 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:26.747048 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:26.747209 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:26.747536 (+ 327us) server_negotiation.cc:299] Negotiation successful
0811 22:09:26.747940 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":163,"threads_started":1}
I20260811 22:09:26.752276 31811 raft_consensus.cc:348] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a670e1d4cd834325adb9f2a5815f8b13" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35917 } }
I20260811 22:09:26.752579 31811 raft_consensus.cc:374] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:26.752672 31811 raft_consensus.cc:729] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a670e1d4cd834325adb9f2a5815f8b13, State: Initialized, Role: FOLLOWER
I20260811 22:09:26.753141 31811 consensus_queue.cc:260] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a670e1d4cd834325adb9f2a5815f8b13" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35917 } }
I20260811 22:09:26.753355 31811 raft_consensus.cc:388] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:26.753443 31811 raft_consensus.cc:482] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:26.753571 31811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:26.754690 31811 raft_consensus.cc:504] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a670e1d4cd834325adb9f2a5815f8b13" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35917 } }
I20260811 22:09:26.755111 31811 leader_election.cc:302] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a670e1d4cd834325adb9f2a5815f8b13; no voters:
I20260811 22:09:26.755528 31811 leader_election.cc:288] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:26.755650 31819 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:26.755935 31819 raft_consensus.cc:686] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [term 1 LEADER]: Becoming Leader. State: Replica: a670e1d4cd834325adb9f2a5815f8b13, State: Running, Role: LEADER
I20260811 22:09:26.756330 31819 consensus_queue.cc:237] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a670e1d4cd834325adb9f2a5815f8b13" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35917 } }
I20260811 22:09:26.756968 31811 sys_catalog.cc:564] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:26.757719 31821 sys_catalog.cc:455] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a670e1d4cd834325adb9f2a5815f8b13. Latest consensus state: current_term: 1 leader_uuid: "a670e1d4cd834325adb9f2a5815f8b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a670e1d4cd834325adb9f2a5815f8b13" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35917 } } }
I20260811 22:09:26.757987 31821 sys_catalog.cc:458] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:26.758147 31820 sys_catalog.cc:455] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a670e1d4cd834325adb9f2a5815f8b13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a670e1d4cd834325adb9f2a5815f8b13" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35917 } } }
I20260811 22:09:26.758265 31820 sys_catalog.cc:458] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:26.758534 31831 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:26.762354 31831 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:26.769124 31831 catalog_manager.cc:1347] Generated new cluster ID: 85b221ff3faa4ffd8ddd2a3ac6dde32a
I20260811 22:09:26.769272 31831 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:26.788090 31831 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:26.789626 31831 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:26.795554 31831 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a670e1d4cd834325adb9f2a5815f8b13: Generated new TSK 0
I20260811 22:09:26.796346 31831 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:09:26.841295 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35917
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:26.967628 31845 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:26.967949 31845 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:26.967996 31845 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:26.972322 31845 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:26.972427 31845 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:26.972491 31845 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:26.978570 31845 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31845
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:26.979846 31845 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:26.981002 31845 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:26.991783 31854 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:26.991758 31852 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:26.991892 31851 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:26.992201 31845 server_base.cc:1034] running on GCE node
I20260811 22:09:26.992682 31845 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:26.993503 31845 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:26.994686 31845 hybrid_clock.cc:648] HybridClock initialized: now 1786486166994671 us; error 31 us; skew 500 ppm
Aug 11 22:09:26 dist-test-slave-vxj2 krb5kdc[31714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486166, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:26.997951 31845 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:09:26.999020 31845 webserver.cc:460] Webserver started at http://127.28.47.129:33795/ using document root <none> and password file <none>
I20260811 22:09:26.999650 31845 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:26.999747 31845 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:26.999997 31845 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:27.002336 31845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "22e10d3276684419933b7f98514dabce"
format_stamp: "Formatted at 2026-08-11 22:09:26 on dist-test-slave-vxj2"
server_key: "08662c5ff136fc77a828f54f9988e74b"
server_key_iv: "d8f9828ec82b860cb9098bc4ca20eb46"
server_key_version: "encryptionkey@0"
I20260811 22:09:27.002965 31845 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "22e10d3276684419933b7f98514dabce"
format_stamp: "Formatted at 2026-08-11 22:09:26 on dist-test-slave-vxj2"
server_key: "08662c5ff136fc77a828f54f9988e74b"
server_key_iv: "d8f9828ec82b860cb9098bc4ca20eb46"
server_key_version: "encryptionkey@0"
I20260811 22:09:27.007306 31845 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:09:27.010315 31862 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:27.011833 31845 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:09:27.012029 31845 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "22e10d3276684419933b7f98514dabce"
format_stamp: "Formatted at 2026-08-11 22:09:26 on dist-test-slave-vxj2"
server_key: "08662c5ff136fc77a828f54f9988e74b"
server_key_iv: "d8f9828ec82b860cb9098bc4ca20eb46"
server_key_version: "encryptionkey@0"
I20260811 22:09:27.012178 31845 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:27.044821 31845 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:27.048334 31845 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:27.048584 31845 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:27.049413 31845 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:27.050412 31845 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:27.050493 31845 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:27.050570 31845 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:27.050617 31845 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:27.061151 31845 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:35769
I20260811 22:09:27.061192 31997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:35769 every 8 connection(s)
I20260811 22:09:27.062202 31845 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:27.069767 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 31845
I20260811 22:09:27.069969 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:09:27.070263 28862 external_mini_cluster.cc:1428] Setting key 224c0675db1cd65d8202df65b3a2cd61
Aug 11 22:09:27 dist-test-slave-vxj2 krb5kdc[31714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486166, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:27.080016 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:27.064399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:52987 (local address 127.28.47.190:35917)
0811 22:09:27.064581 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:27.064586 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:27.065541 (+ 955us) server_negotiation.cc:407] Connection header received
0811 22:09:27.067029 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:27.067032 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:27.067092 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:27.067192 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:27.068604 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:27.069330 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:27.070737 (+ 1407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:27.071040 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:27.071150 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:27.074293 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:27.074297 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:27.074299 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:27.074329 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:27.076595 (+ 2266us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:27.077620 (+ 1025us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:27.077626 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:27.077629 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:27.077873 (+ 244us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:27.078445 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:27.078481 (+ 36us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:27.078486 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:27.078833 (+ 347us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:27.078966 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:27.079609 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 22:09:27.079802 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:09:27.080904 32000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:27.064774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:35917 (local address 127.28.47.129:52987)
0811 22:09:27.065371 (+ 597us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:27.065421 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:27.066716 (+ 1295us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:27.067384 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:27.067391 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:27.067801 (+ 410us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:27.068407 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:27.068425 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:27.069480 (+ 1055us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:27.069483 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:27.070575 (+ 1092us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:27.070586 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:27.071228 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:27.071233 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:27.071450 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:27.072199 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:27.072218 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:27.074015 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:27.076778 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:27.076787 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:27.076832 (+ 45us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:27.077363 (+ 531us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:27.078049 (+ 686us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:27.078054 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:27.078058 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:27.078297 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:27.078976 (+ 679us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:27.078982 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:27.079292 (+ 310us) client_negotiation.cc:769] Sending connection context
0811 22:09:27.079480 (+ 188us) client_negotiation.cc:240] Negotiation successful
0811 22:09:27.079793 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":132,"threads_started":1}
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
I20260811 22:09:27.082402 31998 heartbeater.cc:347] Connected to a master server at 127.28.47.190:35917
I20260811 22:09:27.082737 31998 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:27.083410 31998 heartbeater.cc:511] Master 127.28.47.190:35917 requested a full tablet report, sending...
I20260811 22:09:27.085381 31769 ts_manager.cc:194] Registered new tserver with Master: 22e10d3276684419933b7f98514dabce (127.28.47.129:35769)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:09:27.087117 31769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:52987
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:09:27.105504 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35917
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:27.228433 32003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:27.228816 32003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:27.228910 32003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:27.233243 32003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:27.233366 32003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:27.233465 32003 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:27.239305 32003 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:27.240638 32003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:27.241830 32003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:27.252980 32008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:27.253005 32009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:27.252980 32013 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:27.253304 32003 server_base.cc:1034] running on GCE node
I20260811 22:09:27.253944 32003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:27.254884 32003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:27.256244 32003 hybrid_clock.cc:648] HybridClock initialized: now 1786486167256186 us; error 59 us; skew 500 ppm
Aug 11 22:09:27 dist-test-slave-vxj2 krb5kdc[31714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486167, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:27.260169 32003 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:09:27.261307 32003 webserver.cc:460] Webserver started at http://127.28.47.130:33611/ using document root <none> and password file <none>
I20260811 22:09:27.261963 32003 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:27.262068 32003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:27.262329 32003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:27.264750 32003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "d82d116f7c3e42459f6f7452a6623fcc"
format_stamp: "Formatted at 2026-08-11 22:09:27 on dist-test-slave-vxj2"
server_key: "77c2d8566c7b550f4cbc55edbecd6b07"
server_key_iv: "72bcf85f3d40bd51fa1a6de7e65edae2"
server_key_version: "encryptionkey@0"
I20260811 22:09:27.265450 32003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "d82d116f7c3e42459f6f7452a6623fcc"
format_stamp: "Formatted at 2026-08-11 22:09:27 on dist-test-slave-vxj2"
server_key: "77c2d8566c7b550f4cbc55edbecd6b07"
server_key_iv: "72bcf85f3d40bd51fa1a6de7e65edae2"
server_key_version: "encryptionkey@0"
I20260811 22:09:27.269812 32003 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 22:09:27.272768 32020 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:27.274441 32003 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:09:27.274622 32003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "d82d116f7c3e42459f6f7452a6623fcc"
format_stamp: "Formatted at 2026-08-11 22:09:27 on dist-test-slave-vxj2"
server_key: "77c2d8566c7b550f4cbc55edbecd6b07"
server_key_iv: "72bcf85f3d40bd51fa1a6de7e65edae2"
server_key_version: "encryptionkey@0"
I20260811 22:09:27.274770 32003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:27.290796 32003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:27.294866 32003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:27.295147 32003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:27.295948 32003 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:27.296924 32003 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:27.296975 32003 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:27.297045 32003 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:27.297082 32003 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:27.306932 32003 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:45953
I20260811 22:09:27.306958 32154 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:45953 every 8 connection(s)
I20260811 22:09:27.308274 32003 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:27.314121 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 32003
I20260811 22:09:27.314258 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:09:27.314568 28862 external_mini_cluster.cc:1428] Setting key 5de8f27c46517f2566967fc794e7412d
Aug 11 22:09:27 dist-test-slave-vxj2 krb5kdc[31714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486167, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:27.326086 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:27.310635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:41237 (local address 127.28.47.190:35917)
0811 22:09:27.310847 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:27.310851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:27.312151 (+ 1300us) server_negotiation.cc:407] Connection header received
0811 22:09:27.313260 (+ 1109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:27.313264 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:27.313339 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:27.313432 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:27.315290 (+ 1858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:27.315908 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:27.317548 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:27.317980 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:27.318128 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:27.321631 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:27.321634 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:27.321637 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:27.321691 (+ 54us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:27.323295 (+ 1604us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:27.324042 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:27.324044 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:27.324046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:27.324130 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:27.324615 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:27.324618 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:27.324620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:27.324846 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:27.324943 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:27.325778 (+ 835us) server_negotiation.cc:299] Negotiation successful
0811 22:09:27.325953 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:09:27.327157 32157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:27.311124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:35917 (local address 127.28.47.130:41237)
0811 22:09:27.311953 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:27.312007 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:27.312925 (+ 918us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:27.313633 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:27.313640 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:27.314254 (+ 614us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:27.315075 (+ 821us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:27.315095 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:27.316111 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:27.316115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:27.317399 (+ 1284us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:27.317412 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:27.318189 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:27.318192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:27.318470 (+ 278us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:27.319255 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:27.319283 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:27.321390 (+ 2107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:27.323449 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:27.323459 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:27.323480 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:27.323923 (+ 443us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:27.324229 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:27.324234 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:27.324237 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:27.324469 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:27.324980 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:27.324988 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:27.325441 (+ 453us) client_negotiation.cc:769] Sending connection context
0811 22:09:27.325722 (+ 281us) client_negotiation.cc:240] Negotiation successful
0811 22:09:27.326148 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":596,"thread_start_us":143,"threads_started":1}
I20260811 22:09:27.328626 32155 heartbeater.cc:347] Connected to a master server at 127.28.47.190:35917
I20260811 22:09:27.328874 32155 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
I20260811 22:09:27.329540 32155 heartbeater.cc:511] Master 127.28.47.190:35917 requested a full tablet report, sending...
I20260811 22:09:27.331001 31769 ts_manager.cc:194] Registered new tserver with Master: d82d116f7c3e42459f6f7452a6623fcc (127.28.47.130:45953)
I20260811 22:09:27.331655 31769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:41237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:09:27.349393 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35917
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:37393
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:27.475742 32161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:27.476238 32161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:27.476344 32161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:27.480808 32161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:27.480919 32161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:27.481011 32161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:27.487035 32161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:37393
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:27.488282 32161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:27.489521 32161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:27.500399 32167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:27.500399 32170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:27.500463 32161 server_base.cc:1034] running on GCE node
W20260811 22:09:27.500396 32166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:27.501356 32161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:27.502144 32161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:27.503376 32161 hybrid_clock.cc:648] HybridClock initialized: now 1786486167503358 us; error 29 us; skew 500 ppm
Aug 11 22:09:27 dist-test-slave-vxj2 krb5kdc[31714](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486167, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:27.506886 32161 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:09:27.508296 32161 webserver.cc:460] Webserver started at http://127.28.47.131:44189/ using document root <none> and password file <none>
I20260811 22:09:27.509210 32161 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:27.509321 32161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:27.509626 32161 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:27.512989 32161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "b902b4b54d474dd49c1eb82cd9882a70"
format_stamp: "Formatted at 2026-08-11 22:09:27 on dist-test-slave-vxj2"
server_key: "a560649e37cd4fc738c74476bde57e77"
server_key_iv: "d3f34563a612e10ed352d6cdb759761f"
server_key_version: "encryptionkey@0"
I20260811 22:09:27.513893 32161 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "b902b4b54d474dd49c1eb82cd9882a70"
format_stamp: "Formatted at 2026-08-11 22:09:27 on dist-test-slave-vxj2"
server_key: "a560649e37cd4fc738c74476bde57e77"
server_key_iv: "d3f34563a612e10ed352d6cdb759761f"
server_key_version: "encryptionkey@0"
I20260811 22:09:27.519765 32161 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.005s sys 0.000s
I20260811 22:09:27.523588 32180 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:27.525609 32161 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:09:27.525803 32161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "b902b4b54d474dd49c1eb82cd9882a70"
format_stamp: "Formatted at 2026-08-11 22:09:27 on dist-test-slave-vxj2"
server_key: "a560649e37cd4fc738c74476bde57e77"
server_key_iv: "d3f34563a612e10ed352d6cdb759761f"
server_key_version: "encryptionkey@0"
I20260811 22:09:27.525976 32161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:27.549899 32161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:27.554875 32161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:27.555145 32161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:27.556082 32161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:27.557456 32161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:27.557528 32161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:27.557596 32161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:27.557634 32161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:27.571404 32161 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:36237
I20260811 22:09:27.571435 32318 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:36237 every 8 connection(s)
I20260811 22:09:27.572846 32161 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:27.577890 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 32161
I20260811 22:09:27.578037 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:09:27.578338 28862 external_mini_cluster.cc:1428] Setting key 8f4a4eb41de765ed12ed6e5c97cf545d
Aug 11 22:09:27 dist-test-slave-vxj2 krb5kdc[31714](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486167, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:27.591732 31814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:27.575413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:39963 (local address 127.28.47.190:35917)
0811 22:09:27.575690 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:27.575694 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:27.577037 (+ 1343us) server_negotiation.cc:407] Connection header received
0811 22:09:27.578998 (+ 1961us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:27.579003 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:27.579073 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:27.579161 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:27.580998 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:27.581954 (+ 956us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:27.583235 (+ 1281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:27.583665 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:27.583795 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:27.587007 (+ 3212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:27.587014 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:27.587019 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:27.587059 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:27.588903 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:27.589761 (+ 858us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:27.589767 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:27.589769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:27.589885 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:27.590271 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:27.590276 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:27.590278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:27.590577 (+ 299us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:27.590691 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:27.591325 (+ 634us) server_negotiation.cc:299] Negotiation successful
0811 22:09:27.591503 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:09:27.593017 32321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:27.576164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:35917 (local address 127.28.47.131:39963)
0811 22:09:27.576816 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:27.576867 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:27.578628 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:27.579414 (+ 786us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:27.579426 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:27.580058 (+ 632us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:27.580799 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:27.580819 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:27.582106 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:27.582109 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:27.583074 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:27.583085 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:27.583849 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:27.583852 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:27.584046 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:27.584745 (+ 699us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:27.584764 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:27.586742 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:27.589076 (+ 2334us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:27.589095 (+ 19us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:27.589112 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:27.589623 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:27.590003 (+ 380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:27.590005 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:27.590007 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:27.590164 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:27.590687 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:27.590692 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:27.590983 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 22:09:27.591229 (+ 246us) client_negotiation.cc:240] Negotiation successful
0811 22:09:27.591699 (+ 470us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":428,"thread_start_us":203,"threads_started":1}
I20260811 22:09:27.594331 32319 heartbeater.cc:347] Connected to a master server at 127.28.47.190:35917
I20260811 22:09:27.594681 32319 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:27.595423 32319 heartbeater.cc:511] Master 127.28.47.190:35917 requested a full tablet report, sending...
I20260811 22:09:27.596693 31769 ts_manager.cc:194] Registered new tserver with Master: b902b4b54d474dd49c1eb82cd9882a70 (127.28.47.131:36237)
I20260811 22:09:27.597380 31769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:39963
I20260811 22:09:27.605748 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:09:27.624956 31814 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:27.623305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39842 (local address 127.28.47.190:35917)
0811 22:09:27.623490 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:27.623497 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:27.623625 (+ 128us) server_negotiation.cc:407] Connection header received
0811 22:09:27.623754 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:27.623758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:27.623823 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:27.623942 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:27.624775 (+ 833us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39842: BlockingRecv error: recv got EOF from 127.0.0.1:39842 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:09:27.626149 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 31845
I20260811 22:09:27.635222 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 32003
I20260811 22:09:27.646585 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 32161
I20260811 22:09:27.657361 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 31731
2026-08-11T22:09:27Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2471 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:27 dist-test-slave-vxj2 krb5kdc[32332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:27 dist-test-slave-vxj2 krb5kdc[32332](info): set up 2 sockets
Aug 11 22:09:27 dist-test-slave-vxj2 krb5kdc[32332](info): commencing operation
krb5kdc: starting...
W20260811 22:09:28.805686 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:09:28 dist-test-slave-vxj2 krb5kdc[32332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:09:28.949421 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45537
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:38683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:45537
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:29.071877 32351 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:29.072278 32351 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:29.072379 32351 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:29.076727 32351 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:29.076866 32351 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:29.076913 32351 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:29.076959 32351 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:29.077000 32351 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:29.083209 32351 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:38683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:45537
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45537
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32351
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:29.084890 32351 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:29.086519 32351 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:29.096599 32359 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:29.096599 32361 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:29.096601 32358 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:29.096716 32351 server_base.cc:1034] running on GCE node
I20260811 22:09:29.097784 32351 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:29.099097 32351 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:29.100322 32351 hybrid_clock.cc:648] HybridClock initialized: now 1786486169100287 us; error 63 us; skew 500 ppm
Aug 11 22:09:29 dist-test-slave-vxj2 krb5kdc[32332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486169, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:29.109135 32351 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:09:29.110421 32351 webserver.cc:460] Webserver started at http://127.28.47.190:44123/ using document root <none> and password file <none>
I20260811 22:09:29.111135 32351 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:29.111250 32351 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:29.111532 32351 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:29.114054 32351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "6a4b1655f55545749a2dc3abe0f2bb75"
format_stamp: "Formatted at 2026-08-11 22:09:29 on dist-test-slave-vxj2"
server_key: "36281c13f28236ce355cdfaeb3d92a25"
server_key_iv: "91af5b9ff6e40ecc777b7c52e450ed41"
server_key_version: "encryptionkey@0"
I20260811 22:09:29.114734 32351 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "6a4b1655f55545749a2dc3abe0f2bb75"
format_stamp: "Formatted at 2026-08-11 22:09:29 on dist-test-slave-vxj2"
server_key: "36281c13f28236ce355cdfaeb3d92a25"
server_key_iv: "91af5b9ff6e40ecc777b7c52e450ed41"
server_key_version: "encryptionkey@0"
I20260811 22:09:29.119273 32351 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:09:29.122337 32369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:29.123845 32351 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.004s
I20260811 22:09:29.124074 32351 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "6a4b1655f55545749a2dc3abe0f2bb75"
format_stamp: "Formatted at 2026-08-11 22:09:29 on dist-test-slave-vxj2"
server_key: "36281c13f28236ce355cdfaeb3d92a25"
server_key_iv: "91af5b9ff6e40ecc777b7c52e450ed41"
server_key_version: "encryptionkey@0"
I20260811 22:09:29.124228 32351 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:29.135425 32351 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:29.139118 32351 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:29.139356 32351 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:29.147122 32351 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:45537
I20260811 22:09:29.147580 32427 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:45537 every 8 connection(s)
I20260811 22:09:29.148260 32351 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:29.151458 32428 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:29.157621 32428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75: Bootstrap starting.
I20260811 22:09:29.158059 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 32351
I20260811 22:09:29.158185 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:29.158486 28862 external_mini_cluster.cc:1428] Setting key 1c023639d8a81ce41f76f58499f3000f
I20260811 22:09:29.160663 32428 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:29.161545 32428 log.cc:824] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:29.163918 32428 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75: No bootstrap required, opened a new log
Aug 11 22:09:29 dist-test-slave-vxj2 krb5kdc[32332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486168, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:29.169015 32428 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4b1655f55545749a2dc3abe0f2bb75" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45537 } }
I20260811 22:09:29.169409 32428 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:29.169500 32428 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a4b1655f55545749a2dc3abe0f2bb75, State: Initialized, Role: FOLLOWER
I20260811 22:09:29.170042 32428 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4b1655f55545749a2dc3abe0f2bb75" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45537 } }
I20260811 22:09:29.170207 32428 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:29.170301 32428 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:29.170411 32428 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:29.171422 32428 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4b1655f55545749a2dc3abe0f2bb75" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45537 } }
I20260811 22:09:29.171877 32428 leader_election.cc:302] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6a4b1655f55545749a2dc3abe0f2bb75; no voters:
I20260811 22:09:29.172344 32428 leader_election.cc:288] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:29.172456 32433 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:29.172713 32433 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [term 1 LEADER]: Becoming Leader. State: Replica: 6a4b1655f55545749a2dc3abe0f2bb75, State: Running, Role: LEADER
I20260811 22:09:29.173081 32433 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4b1655f55545749a2dc3abe0f2bb75" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45537 } }
I20260811 22:09:29.173997 32428 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:29.175380 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:29.159893 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51022 (local address 127.28.47.190:45537)
0811 22:09:29.160485 (+ 592us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:29.160506 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:29.160556 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:09:29.161394 (+ 838us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:29.161415 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:29.161813 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:29.162209 (+ 396us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:29.163096 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:29.164154 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:29.165415 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:29.165909 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:29.166037 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:29.169289 (+ 3252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:29.169295 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:29.169312 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:29.169356 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:29.171409 (+ 2053us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:29.172026 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:29.172035 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:29.172044 (+ 9us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:29.172193 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:29.172570 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:29.172574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:29.172578 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:29.173191 (+ 613us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:29.173392 (+ 201us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:29.173971 (+ 579us) server_negotiation.cc:299] Negotiation successful
0811 22:09:29.174332 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":147,"threads_started":1}
I20260811 22:09:29.176590 32434 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6a4b1655f55545749a2dc3abe0f2bb75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4b1655f55545749a2dc3abe0f2bb75" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45537 } } }
I20260811 22:09:29.177276 32434 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:29.177656 32435 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6a4b1655f55545749a2dc3abe0f2bb75. Latest consensus state: current_term: 1 leader_uuid: "6a4b1655f55545749a2dc3abe0f2bb75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6a4b1655f55545749a2dc3abe0f2bb75" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45537 } } }
I20260811 22:09:29.177835 32435 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:29.179086 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260811 22:09:29.179658 32449 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:09:29.179747 32449 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:09:29.179837 32452 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:29.180784 32452 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:29.188473 32452 catalog_manager.cc:1347] Generated new cluster ID: 84320d42c6a34349a1d9c5bd12c1b3de
I20260811 22:09:29.188577 32452 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:29.199542 32431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:29.194362 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51038 (local address 127.28.47.190:45537)
0811 22:09:29.194519 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:29.194524 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:29.194770 (+ 246us) server_negotiation.cc:407] Connection header received
0811 22:09:29.194951 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:29.194954 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:29.195010 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:29.195098 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:29.195670 (+ 572us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:29.196220 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:29.197101 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:29.197447 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:29.197562 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:29.198507 (+ 945us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:29.198511 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:29.198514 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:29.199027 (+ 513us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:29.199060 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:29.199077 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:29.199180 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:29.199306 (+ 126us) server_negotiation.cc:299] Negotiation successful
0811 22:09:29.199376 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:09:29.210620 32452 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:29.212035 32452 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:29.223552 32452 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6a4b1655f55545749a2dc3abe0f2bb75: Generated new TSK 0
I20260811 22:09:29.224570 32452 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:09:29.231539 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 32351
2026-08-11T22:09:29Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1573 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:29 dist-test-slave-vxj2 krb5kdc[32465](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:29 dist-test-slave-vxj2 krb5kdc[32465](info): set up 2 sockets
Aug 11 22:09:29 dist-test-slave-vxj2 krb5kdc[32465](info): commencing operation
krb5kdc: starting...
W20260811 22:09:30.395419 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:09:30 dist-test-slave-vxj2 krb5kdc[32465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:30Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:30Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:09:30.537971 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:38905
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:38905
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:30.659667 32478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:30.660053 32478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:30.660140 32478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:30.664551 32478 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:30.664680 32478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:30.664711 32478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:30.664740 32478 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:30.664827 32478 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:30.670785 32478 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:38905
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:38905
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:30.672195 32478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:30.673460 32478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:30.683736 32486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:30.683736 32485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:30.683894 32489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:30.684201 32478 server_base.cc:1034] running on GCE node
I20260811 22:09:30.684996 32478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:30.686420 32478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:30.687640 32478 hybrid_clock.cc:648] HybridClock initialized: now 1786486170687604 us; error 48 us; skew 500 ppm
Aug 11 22:09:30 dist-test-slave-vxj2 krb5kdc[32465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486170, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:30.691375 32478 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:09:30.692498 32478 webserver.cc:460] Webserver started at http://127.28.47.190:37153/ using document root <none> and password file <none>
I20260811 22:09:30.693256 32478 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:30.693364 32478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:30.693634 32478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:30.695971 32478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "2d7f1edc05ea4e1f9d60bf79c832bf50"
format_stamp: "Formatted at 2026-08-11 22:09:30 on dist-test-slave-vxj2"
server_key: "c664443a009212d46933cb7044d0dcd8"
server_key_iv: "61506811286508f79e1be3f7fc621df6"
server_key_version: "encryptionkey@0"
I20260811 22:09:30.696593 32478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "2d7f1edc05ea4e1f9d60bf79c832bf50"
format_stamp: "Formatted at 2026-08-11 22:09:30 on dist-test-slave-vxj2"
server_key: "c664443a009212d46933cb7044d0dcd8"
server_key_iv: "61506811286508f79e1be3f7fc621df6"
server_key_version: "encryptionkey@0"
I20260811 22:09:30.700901 32478 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260811 22:09:30.703840 32498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:30.705514 32478 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:30.705731 32478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "2d7f1edc05ea4e1f9d60bf79c832bf50"
format_stamp: "Formatted at 2026-08-11 22:09:30 on dist-test-slave-vxj2"
server_key: "c664443a009212d46933cb7044d0dcd8"
server_key_iv: "61506811286508f79e1be3f7fc621df6"
server_key_version: "encryptionkey@0"
I20260811 22:09:30.705881 32478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:30.723394 32478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:30.727095 32478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:30.727329 32478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:30.735383 32478 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:38905
I20260811 22:09:30.735720 32560 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:38905 every 8 connection(s)
I20260811 22:09:30.736593 32478 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:30.740193 32562 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:30.746587 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 32478
I20260811 22:09:30.746711 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:30.746786 32562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50: Bootstrap starting.
I20260811 22:09:30.747076 28862 external_mini_cluster.cc:1428] Setting key ec4e6e102ab838fe4319e15a6efaf6f2
I20260811 22:09:30.749989 32562 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:30.750962 32562 log.cc:824] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:30.753631 32562 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50: No bootstrap required, opened a new log
Aug 11 22:09:30 dist-test-slave-vxj2 krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:30.758108 32562 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7f1edc05ea4e1f9d60bf79c832bf50" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38905 } }
I20260811 22:09:30.758510 32562 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:30.758605 32562 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2d7f1edc05ea4e1f9d60bf79c832bf50, State: Initialized, Role: FOLLOWER
I20260811 22:09:30.759109 32562 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7f1edc05ea4e1f9d60bf79c832bf50" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38905 } }
I20260811 22:09:30.759279 32562 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:30.759385 32562 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:30.759526 32562 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:30.760560 32562 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7f1edc05ea4e1f9d60bf79c832bf50" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38905 } }
I20260811 22:09:30.760960 32562 leader_election.cc:302] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2d7f1edc05ea4e1f9d60bf79c832bf50; no voters:
I20260811 22:09:30.761423 32562 leader_election.cc:288] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:30.761552 32569 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:30.761755 32569 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [term 1 LEADER]: Becoming Leader. State: Replica: 2d7f1edc05ea4e1f9d60bf79c832bf50, State: Running, Role: LEADER
I20260811 22:09:30.762111 32569 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7f1edc05ea4e1f9d60bf79c832bf50" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38905 } }
I20260811 22:09:30.762938 32562 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:30.763917 32570 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2d7f1edc05ea4e1f9d60bf79c832bf50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7f1edc05ea4e1f9d60bf79c832bf50" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38905 } } }
I20260811 22:09:30.764053 32570 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:30.764381 32572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2d7f1edc05ea4e1f9d60bf79c832bf50. Latest consensus state: current_term: 1 leader_uuid: "2d7f1edc05ea4e1f9d60bf79c832bf50" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2d7f1edc05ea4e1f9d60bf79c832bf50" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38905 } } }
I20260811 22:09:30.764460 32580 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:30.764474 32572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:30.765432 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:30.748668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37054 (local address 127.28.47.190:38905)
0811 22:09:30.749229 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:30.749250 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:30.749299 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:09:30.750326 (+ 1027us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:30.750354 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:30.750822 (+ 468us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:30.751246 (+ 424us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:30.752349 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:30.753822 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:30.754829 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:30.755582 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:30.755786 (+ 204us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:30.758784 (+ 2998us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:30.758790 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:30.758807 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:30.758849 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:30.760933 (+ 2084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:30.761541 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:30.761548 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:30.761555 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:30.761757 (+ 202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:30.762107 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:30.762111 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:30.762115 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:30.762700 (+ 585us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:30.762954 (+ 254us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:30.763771 (+ 817us) server_negotiation.cc:299] Negotiation successful
0811 22:09:30.764139 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":133,"threads_started":1}
I20260811 22:09:30.768524 32580 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:30.775058 32580 catalog_manager.cc:1347] Generated new cluster ID: 538e6a3feb9b4431a8c4b0770d210e08
I20260811 22:09:30.775154 32580 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:30.782917 32580 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:30.784018 32580 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:30.792590 32580 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2d7f1edc05ea4e1f9d60bf79c832bf50: Generated new TSK 0
I20260811 22:09:30.793656 32580 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:09:30.837838 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38905
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:30.963551 32592 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:30.963965 32592 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:30.964040 32592 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:30.968470 32592 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:30.968544 32592 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:30.968607 32592 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:30.974399 32592 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32592
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:30.975649 32592 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:30.976711 32592 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:30.987699 32598 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:30.987684 32597 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:30.987759 32592 server_base.cc:1034] running on GCE node
W20260811 22:09:30.987681 32602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:30.988446 32592 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:30.989357 32592 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:30.990585 32592 hybrid_clock.cc:648] HybridClock initialized: now 1786486170990565 us; error 34 us; skew 500 ppm
Aug 11 22:09:30 dist-test-slave-vxj2 krb5kdc[32465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486170, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:30.993868 32592 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:09:30.994908 32592 webserver.cc:460] Webserver started at http://127.28.47.129:34173/ using document root <none> and password file <none>
I20260811 22:09:30.995493 32592 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:30.995613 32592 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:30.995843 32592 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:30.998534 32592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "5223eda344064740af589bb94bc71ec2"
format_stamp: "Formatted at 2026-08-11 22:09:30 on dist-test-slave-vxj2"
server_key: "c5499bd0c5d90ff04414dcf84ddf890a"
server_key_iv: "d5fbffb56990044c190748d6c3030c5c"
server_key_version: "encryptionkey@0"
I20260811 22:09:30.999235 32592 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "5223eda344064740af589bb94bc71ec2"
format_stamp: "Formatted at 2026-08-11 22:09:30 on dist-test-slave-vxj2"
server_key: "c5499bd0c5d90ff04414dcf84ddf890a"
server_key_iv: "d5fbffb56990044c190748d6c3030c5c"
server_key_version: "encryptionkey@0"
I20260811 22:09:31.004128 32592 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:09:31.007359 32609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:31.008955 32592 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:09:31.009124 32592 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "5223eda344064740af589bb94bc71ec2"
format_stamp: "Formatted at 2026-08-11 22:09:30 on dist-test-slave-vxj2"
server_key: "c5499bd0c5d90ff04414dcf84ddf890a"
server_key_iv: "d5fbffb56990044c190748d6c3030c5c"
server_key_version: "encryptionkey@0"
I20260811 22:09:31.009295 32592 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:31.024286 32592 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:31.027966 32592 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:31.028196 32592 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:31.028890 32592 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:31.029820 32592 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:31.029892 32592 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:31.029969 32592 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:31.030014 32592 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:31.039891 32592 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:44011
I20260811 22:09:31.039919 32751 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:44011 every 8 connection(s)
I20260811 22:09:31.040864 32592 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:31.046785 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 32592
I20260811 22:09:31.046911 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:09:31.047286 28862 external_mini_cluster.cc:1428] Setting key ef63b1faeff325da6e3ef6d267f5a320
Aug 11 22:09:31 dist-test-slave-vxj2 krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486170, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:31.057118 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:31.043030 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:34851 (local address 127.28.47.190:38905)
0811 22:09:31.043181 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:31.043185 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:31.044134 (+ 949us) server_negotiation.cc:407] Connection header received
0811 22:09:31.045914 (+ 1780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:31.045917 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:31.045989 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:31.046075 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:31.047644 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.048317 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.049664 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.049919 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.050040 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:31.052829 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:31.052833 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:31.052835 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:31.052864 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:31.054698 (+ 1834us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:31.055335 (+ 637us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:31.055340 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:31.055342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:31.055446 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:31.055840 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:31.055844 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:31.055846 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:31.056065 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:31.056182 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:31.056781 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 22:09:31.056938 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
I20260811 22:09:31.058224 32754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:31.043402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38905 (local address 127.28.47.129:34851)
0811 22:09:31.043953 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:31.044002 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:31.045624 (+ 1622us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:31.046276 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:31.046291 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:31.046722 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:31.047460 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:31.047478 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.048461 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.048471 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:31.049527 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:31.049539 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.050095 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.050099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:31.050286 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:31.050971 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:31.050990 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:31.052612 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:31.054865 (+ 2253us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:31.054872 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:31.054894 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:31.055211 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:31.055549 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:31.055552 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:31.055554 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:31.055737 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:31.056183 (+ 446us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:31.056192 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:31.056512 (+ 320us) client_negotiation.cc:769] Sending connection context
0811 22:09:31.056729 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 22:09:31.057106 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":351,"thread_start_us":125,"threads_started":1}
I20260811 22:09:31.059890 32752 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38905
I20260811 22:09:31.060262 32752 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:31.060959 32752 heartbeater.cc:511] Master 127.28.47.190:38905 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:09:31.063166 32519 ts_manager.cc:194] Registered new tserver with Master: 5223eda344064740af589bb94bc71ec2 (127.28.47.129:44011)
I20260811 22:09:31.064687 32519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:34851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:09:31.082564 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38905
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:31.209574 32757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:31.209894 32757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:31.209975 32757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:31.214666 32757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:31.214753 32757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:31.214820 32757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:31.220741 32757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:31.222057 32757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:31.223173 32757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:31.234592 32767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:31.234555 32762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:31.234555 32763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:31.234624 32757 server_base.cc:1034] running on GCE node
I20260811 22:09:31.235447 32757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:31.236279 32757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:31.237533 32757 hybrid_clock.cc:648] HybridClock initialized: now 1786486171237506 us; error 44 us; skew 500 ppm
Aug 11 22:09:31 dist-test-slave-vxj2 krb5kdc[32465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486171, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:31.241086 32757 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:09:31.242211 32757 webserver.cc:460] Webserver started at http://127.28.47.130:41891/ using document root <none> and password file <none>
I20260811 22:09:31.242925 32757 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:31.243028 32757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:31.243296 32757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:31.245709 32757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "85b8512f8a0b416aa9125aacb7ddc2f7"
format_stamp: "Formatted at 2026-08-11 22:09:31 on dist-test-slave-vxj2"
server_key: "affdeec668ad33e3deaa7ba9099dda01"
server_key_iv: "272e8bbb2a9369d0615e496ace304a4e"
server_key_version: "encryptionkey@0"
I20260811 22:09:31.246374 32757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "85b8512f8a0b416aa9125aacb7ddc2f7"
format_stamp: "Formatted at 2026-08-11 22:09:31 on dist-test-slave-vxj2"
server_key: "affdeec668ad33e3deaa7ba9099dda01"
server_key_iv: "272e8bbb2a9369d0615e496ace304a4e"
server_key_version: "encryptionkey@0"
I20260811 22:09:31.250826 32757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:09:31.253856 309 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:31.255470 32757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:31.255652 32757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "85b8512f8a0b416aa9125aacb7ddc2f7"
format_stamp: "Formatted at 2026-08-11 22:09:31 on dist-test-slave-vxj2"
server_key: "affdeec668ad33e3deaa7ba9099dda01"
server_key_iv: "272e8bbb2a9369d0615e496ace304a4e"
server_key_version: "encryptionkey@0"
I20260811 22:09:31.255797 32757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:31.269398 32757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:31.273138 32757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:31.273461 32757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:31.274164 32757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:31.275063 32757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:31.275110 32757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:31.275163 32757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:31.275185 32757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:31.284984 32757 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:34187
I20260811 22:09:31.285009 447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:34187 every 8 connection(s)
I20260811 22:09:31.286166 32757 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:31.291939 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 32757
I20260811 22:09:31.292086 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:09:31.292378 28862 external_mini_cluster.cc:1428] Setting key 85d7c4ec428719c9f480518323b7f02b
Aug 11 22:09:31 dist-test-slave-vxj2 krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486171, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:31.301754 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:31.288333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:47241 (local address 127.28.47.190:38905)
0811 22:09:31.288495 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:31.288499 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:31.289545 (+ 1046us) server_negotiation.cc:407] Connection header received
0811 22:09:31.291042 (+ 1497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:31.291046 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:31.291115 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:31.291237 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:31.292746 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.293453 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.294896 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.295128 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.295264 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:31.297998 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:31.298002 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:31.298004 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:31.298029 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:31.299521 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:31.300074 (+ 553us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:31.300079 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:31.300081 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:31.300166 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:31.300533 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:31.300538 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:31.300540 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:31.300746 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:31.300858 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:31.301442 (+ 584us) server_negotiation.cc:299] Negotiation successful
0811 22:09:31.301583 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:09:31.302762 450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:31.288802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38905 (local address 127.28.47.130:47241)
0811 22:09:31.289390 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:31.289437 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:31.290805 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:31.291416 (+ 611us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:31.291422 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:31.291844 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:31.292586 (+ 742us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:31.292608 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.293586 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.293590 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:31.294753 (+ 1163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:31.294763 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.295292 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.295296 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:31.295499 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:31.296151 (+ 652us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:31.296169 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:31.297803 (+ 1634us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:31.299653 (+ 1850us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:31.299659 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:31.299678 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:31.299968 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:31.300272 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:31.300275 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:31.300277 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:31.300438 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:31.300857 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:31.300866 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:31.301142 (+ 276us) client_negotiation.cc:769] Sending connection context
0811 22:09:31.301400 (+ 258us) client_negotiation.cc:240] Negotiation successful
0811 22:09:31.301777 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":125,"threads_started":1}
I20260811 22:09:31.304128 448 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38905
I20260811 22:09:31.304394 448 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:31.305004 448 heartbeater.cc:511] Master 127.28.47.190:38905 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
I20260811 22:09:31.306298 32519 ts_manager.cc:194] Registered new tserver with Master: 85b8512f8a0b416aa9125aacb7ddc2f7 (127.28.47.130:34187)
I20260811 22:09:31.306979 32519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:47241
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:09:31.326272 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38905
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:31.453248 453 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:31.453557 453 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:31.453605 453 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:31.458043 453 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:31.458132 453 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:31.458228 453 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:31.464593 453 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38905
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.453
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:31.465952 453 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:31.467144 453 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:31.478194 462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:31.478194 461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:31.478192 464 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:31.478631 453 server_base.cc:1034] running on GCE node
I20260811 22:09:31.479162 453 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:31.479972 453 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:31.481231 453 hybrid_clock.cc:648] HybridClock initialized: now 1786486171481195 us; error 52 us; skew 500 ppm
Aug 11 22:09:31 dist-test-slave-vxj2 krb5kdc[32465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486171, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:31.484277 453 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:09:31.485339 453 webserver.cc:460] Webserver started at http://127.28.47.131:42809/ using document root <none> and password file <none>
I20260811 22:09:31.486176 453 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:31.486281 453 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:31.486578 453 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:31.488700 453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "cf0c5228a9024027b0d70a20482a6d63"
format_stamp: "Formatted at 2026-08-11 22:09:31 on dist-test-slave-vxj2"
server_key: "a917a62fb33a9df73aa903fdbe33fee3"
server_key_iv: "4b28f32de460e805d3df5aeba7ab335f"
server_key_version: "encryptionkey@0"
I20260811 22:09:31.489336 453 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "cf0c5228a9024027b0d70a20482a6d63"
format_stamp: "Formatted at 2026-08-11 22:09:31 on dist-test-slave-vxj2"
server_key: "a917a62fb33a9df73aa903fdbe33fee3"
server_key_iv: "4b28f32de460e805d3df5aeba7ab335f"
server_key_version: "encryptionkey@0"
I20260811 22:09:31.493448 453 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:09:31.496245 476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:31.497821 453 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:31.498005 453 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "cf0c5228a9024027b0d70a20482a6d63"
format_stamp: "Formatted at 2026-08-11 22:09:31 on dist-test-slave-vxj2"
server_key: "a917a62fb33a9df73aa903fdbe33fee3"
server_key_iv: "4b28f32de460e805d3df5aeba7ab335f"
server_key_version: "encryptionkey@0"
I20260811 22:09:31.498135 453 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:31.512655 453 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:31.516211 453 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:31.516434 453 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:31.517102 453 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:31.518064 453 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:31.518113 453 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:31.518157 453 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:31.518172 453 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:31.529002 453 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:34219
I20260811 22:09:31.529019 614 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:34219 every 8 connection(s)
I20260811 22:09:31.530057 453 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:31.534852 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 453
I20260811 22:09:31.535004 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:09:31.535315 28862 external_mini_cluster.cc:1428] Setting key 833d8c059910b7dd108329d79419d4c9
Aug 11 22:09:31 dist-test-slave-vxj2 krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486171, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:31.545817 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:31.532169 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:35115 (local address 127.28.47.190:38905)
0811 22:09:31.532369 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:31.532375 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:31.533280 (+ 905us) server_negotiation.cc:407] Connection header received
0811 22:09:31.534711 (+ 1431us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:31.534718 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:31.534788 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:31.534897 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:31.536526 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.537287 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.538650 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.538885 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.538974 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:31.541845 (+ 2871us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:31.541849 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:31.541851 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:31.541892 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:31.543446 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:31.544042 (+ 596us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:31.544046 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:31.544048 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:31.544137 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:31.544526 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:31.544530 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:31.544532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:31.544763 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:31.544877 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:31.545505 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 22:09:31.545642 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:09:31.546847 619 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:31.532570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38905 (local address 127.28.47.131:35115)
0811 22:09:31.533073 (+ 503us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:31.533118 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:31.534462 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:31.535190 (+ 728us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:31.535198 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:31.535675 (+ 477us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:31.536346 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:31.536391 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.537469 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.537473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:31.538528 (+ 1055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:31.538539 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.539036 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.539039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:31.539194 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:31.539880 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:31.539898 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:31.541620 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:31.543589 (+ 1969us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:31.543595 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:31.543613 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:31.543916 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:31.544233 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:31.544238 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:31.544243 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:31.544407 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:31.544877 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:31.544886 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:31.545244 (+ 358us) client_negotiation.cc:769] Sending connection context
0811 22:09:31.545461 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 22:09:31.545848 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":137,"threads_started":1}
I20260811 22:09:31.548274 616 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38905
I20260811 22:09:31.548556 616 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:31.549299 616 heartbeater.cc:511] Master 127.28.47.190:38905 requested a full tablet report, sending...
I20260811 22:09:31.550554 32519 ts_manager.cc:194] Registered new tserver with Master: cf0c5228a9024027b0d70a20482a6d63 (127.28.47.131:34219)
I20260811 22:09:31.551411 32519 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:35115
I20260811 22:09:31.562235 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:09:31 dist-test-slave-vxj2 krb5kdc[32465](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486171, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 11 22:09:31 dist-test-slave-vxj2 krb5kdc[32465](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486171, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:31.592630 32566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:31.582503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37056 (local address 127.28.47.190:38905)
0811 22:09:31.582651 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:31.582655 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:31.582797 (+ 142us) server_negotiation.cc:407] Connection header received
0811 22:09:31.582870 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:31.582880 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:31.582934 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:31.583025 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:31.584086 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.584839 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.585952 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:31.586244 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:31.586377 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:31.589356 (+ 2979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:31.589360 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:31.589361 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:31.589399 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:31.590823 (+ 1424us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:31.591304 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:31.591306 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:31.591308 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:31.591410 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:31.591676 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:31.591679 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:31.591680 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:31.591943 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:31.592060 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:31.592370 (+ 310us) server_negotiation.cc:299] Negotiation successful
0811 22:09:31.592501 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
W20260811 22:09:31.593252 32519 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:37056
I20260811 22:09:31.594291 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 32592
I20260811 22:09:31.604604 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 32757
I20260811 22:09:31.617766 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 453
I20260811 22:09:31.628262 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 32478
2026-08-11T22:09:31Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2396 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:31 dist-test-slave-vxj2 krb5kdc[630](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:31 dist-test-slave-vxj2 krb5kdc[630](info): set up 2 sockets
Aug 11 22:09:31 dist-test-slave-vxj2 krb5kdc[630](info): commencing operation
krb5kdc: starting...
W20260811 22:09:32.827709 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:09:32 dist-test-slave-vxj2 krb5kdc[630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:09:32.968850 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:42981
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:42981
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:33.095027 647 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:33.095477 647 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:33.095594 647 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:33.101038 647 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:33.101181 647 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:33.101215 647 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:33.101240 647 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:33.101274 647 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:33.107841 647 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:42981
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:42981
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.647
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:33.109133 647 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:33.110424 647 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:33.119982 655 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:33.119989 652 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:33.120193 653 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:33.120397 647 server_base.cc:1034] running on GCE node
I20260811 22:09:33.121125 647 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:33.122524 647 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:33.123739 647 hybrid_clock.cc:648] HybridClock initialized: now 1786486173123701 us; error 50 us; skew 500 ppm
Aug 11 22:09:33 dist-test-slave-vxj2 krb5kdc[630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486173, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:33.127246 647 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:09:33.128495 647 webserver.cc:460] Webserver started at http://127.28.47.190:46563/ using document root <none> and password file <none>
I20260811 22:09:33.129099 647 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:33.129231 647 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:33.129451 647 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:33.131733 647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "e8fab6e9e857401b9dd23a0fcf04923d"
format_stamp: "Formatted at 2026-08-11 22:09:33 on dist-test-slave-vxj2"
server_key: "5499c4a8829912c8f1b9f825db27a5e3"
server_key_iv: "b7e581375cc049ff8254b4ae4625e745"
server_key_version: "encryptionkey@0"
I20260811 22:09:33.132351 647 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "e8fab6e9e857401b9dd23a0fcf04923d"
format_stamp: "Formatted at 2026-08-11 22:09:33 on dist-test-slave-vxj2"
server_key: "5499c4a8829912c8f1b9f825db27a5e3"
server_key_iv: "b7e581375cc049ff8254b4ae4625e745"
server_key_version: "encryptionkey@0"
I20260811 22:09:33.136657 647 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260811 22:09:33.139586 663 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:33.141093 647 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:33.141319 647 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "e8fab6e9e857401b9dd23a0fcf04923d"
format_stamp: "Formatted at 2026-08-11 22:09:33 on dist-test-slave-vxj2"
server_key: "5499c4a8829912c8f1b9f825db27a5e3"
server_key_iv: "b7e581375cc049ff8254b4ae4625e745"
server_key_version: "encryptionkey@0"
I20260811 22:09:33.141453 647 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:33.159471 647 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:33.163619 647 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:33.163832 647 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:33.172456 647 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:42981
I20260811 22:09:33.172483 723 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:42981 every 8 connection(s)
I20260811 22:09:33.173663 647 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:33.177100 724 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:33.177675 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 647
I20260811 22:09:33.177814 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:33.178146 28862 external_mini_cluster.cc:1428] Setting key 7eb3ee82a8b338e2db93d20ff10d8fc9
I20260811 22:09:33.185007 724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d: Bootstrap starting.
I20260811 22:09:33.188009 724 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d: Neither blocks nor log segments found. Creating new log.
Aug 11 22:09:33 dist-test-slave-vxj2 krb5kdc[630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:33.188922 724 log.cc:824] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:33.191669 724 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d: No bootstrap required, opened a new log
I20260811 22:09:33.194520 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:33.179818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58762 (local address 127.28.47.190:42981)
0811 22:09:33.180357 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:33.180373 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:33.180412 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:09:33.181134 (+ 722us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:33.181211 (+ 77us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:33.181707 (+ 496us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:33.182154 (+ 447us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:33.183065 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:33.184152 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:33.185078 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:33.185693 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:33.185896 (+ 203us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:33.188925 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:33.188931 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:33.188947 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:33.188984 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:33.191042 (+ 2058us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:33.191575 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:33.191581 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:33.191586 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:33.191722 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:33.192140 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:33.192142 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:33.192143 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:33.192601 (+ 458us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:33.192769 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:33.193237 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 22:09:33.193608 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":374,"thread_start_us":167,"threads_started":1}
I20260811 22:09:33.196087 724 raft_consensus.cc:348] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fab6e9e857401b9dd23a0fcf04923d" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42981 } }
I20260811 22:09:33.196345 724 raft_consensus.cc:374] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:33.196422 724 raft_consensus.cc:729] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e8fab6e9e857401b9dd23a0fcf04923d, State: Initialized, Role: FOLLOWER
I20260811 22:09:33.196995 724 consensus_queue.cc:260] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fab6e9e857401b9dd23a0fcf04923d" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42981 } }
I20260811 22:09:33.197198 724 raft_consensus.cc:388] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:33.197269 724 raft_consensus.cc:482] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:33.197386 724 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:33.198493 724 raft_consensus.cc:504] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fab6e9e857401b9dd23a0fcf04923d" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42981 } }
I20260811 22:09:33.198890 724 leader_election.cc:302] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e8fab6e9e857401b9dd23a0fcf04923d; no voters:
I20260811 22:09:33.199250 724 leader_election.cc:288] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:33.199450 730 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:33.199688 730 raft_consensus.cc:686] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [term 1 LEADER]: Becoming Leader. State: Replica: e8fab6e9e857401b9dd23a0fcf04923d, State: Running, Role: LEADER
I20260811 22:09:33.200067 730 consensus_queue.cc:237] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fab6e9e857401b9dd23a0fcf04923d" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42981 } }
I20260811 22:09:33.200816 724 sys_catalog.cc:564] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:33.202085 731 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e8fab6e9e857401b9dd23a0fcf04923d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fab6e9e857401b9dd23a0fcf04923d" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42981 } } }
I20260811 22:09:33.202090 733 sys_catalog.cc:455] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e8fab6e9e857401b9dd23a0fcf04923d. Latest consensus state: current_term: 1 leader_uuid: "e8fab6e9e857401b9dd23a0fcf04923d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e8fab6e9e857401b9dd23a0fcf04923d" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42981 } } }
I20260811 22:09:33.202248 731 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:33.202256 733 sys_catalog.cc:458] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:33.202684 740 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:33.206060 740 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:33.212661 740 catalog_manager.cc:1347] Generated new cluster ID: 4cca79664a044b89adbb02d71ec2fc9d
I20260811 22:09:33.212765 740 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:33.246039 740 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:33.247277 740 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:33.256138 740 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e8fab6e9e857401b9dd23a0fcf04923d: Generated new TSK 0
I20260811 22:09:33.257282 740 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:09:33.301908 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42981
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:33.433562 755 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:33.433928 755 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:33.434015 755 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:33.438470 755 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:33.438586 755 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:33.438700 755 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:33.444612 755 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.755
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:33.446012 755 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:33.447255 755 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:33.458424 764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:33.458412 763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:33.458426 767 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:33.458714 755 server_base.cc:1034] running on GCE node
I20260811 22:09:33.459343 755 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:33.460144 755 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:33.461335 755 hybrid_clock.cc:648] HybridClock initialized: now 1786486173461297 us; error 44 us; skew 500 ppm
Aug 11 22:09:33 dist-test-slave-vxj2 krb5kdc[630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486173, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:33.464921 755 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:09:33.466785 755 webserver.cc:460] Webserver started at http://127.28.47.129:33449/ using document root <none> and password file <none>
I20260811 22:09:33.467523 755 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:33.467602 755 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:33.467850 755 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:33.470628 755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "6a3e312d4cca4c948e3087127fc1dd2d"
format_stamp: "Formatted at 2026-08-11 22:09:33 on dist-test-slave-vxj2"
server_key: "6180676d0a7d9e39b6a8221bd3c4c243"
server_key_iv: "6e1b1138fb69eef1cdbac215a3f3b9b6"
server_key_version: "encryptionkey@0"
I20260811 22:09:33.471421 755 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "6a3e312d4cca4c948e3087127fc1dd2d"
format_stamp: "Formatted at 2026-08-11 22:09:33 on dist-test-slave-vxj2"
server_key: "6180676d0a7d9e39b6a8221bd3c4c243"
server_key_iv: "6e1b1138fb69eef1cdbac215a3f3b9b6"
server_key_version: "encryptionkey@0"
I20260811 22:09:33.476303 755 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:09:33.479481 778 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:33.480957 755 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:09:33.481132 755 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "6a3e312d4cca4c948e3087127fc1dd2d"
format_stamp: "Formatted at 2026-08-11 22:09:33 on dist-test-slave-vxj2"
server_key: "6180676d0a7d9e39b6a8221bd3c4c243"
server_key_iv: "6e1b1138fb69eef1cdbac215a3f3b9b6"
server_key_version: "encryptionkey@0"
I20260811 22:09:33.481333 755 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:33.492405 755 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:33.496492 755 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:33.496868 755 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:33.497725 755 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:33.498775 755 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:33.498833 755 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:33.498880 755 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:33.498895 755 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:33.509519 755 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:36549
I20260811 22:09:33.509574 911 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:36549 every 8 connection(s)
I20260811 22:09:33.510636 755 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:33.511145 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 755
I20260811 22:09:33.511322 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:09:33.511801 28862 external_mini_cluster.cc:1428] Setting key 4baa4d472057b4139c820831f9eee869
Aug 11 22:09:33 dist-test-slave-vxj2 krb5kdc[630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486173, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:09:33.529457 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:33.513124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:48317 (local address 127.28.47.190:42981)
0811 22:09:33.513321 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:33.513326 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:33.514522 (+ 1196us) server_negotiation.cc:407] Connection header received
0811 22:09:33.516315 (+ 1793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:33.516323 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:33.516462 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:33.516835 (+ 373us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:33.518314 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:33.518857 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:33.520209 (+ 1352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:33.520470 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:33.520587 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:33.524267 (+ 3680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:33.524271 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:33.524274 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:33.524324 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:33.526572 (+ 2248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:33.527431 (+ 859us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:33.527434 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:33.527435 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:33.527538 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:33.527972 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:33.527976 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:33.527977 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:33.528237 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:33.528380 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:33.529033 (+ 653us) server_negotiation.cc:299] Negotiation successful
0811 22:09:33.529253 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94}
I20260811 22:09:33.530376 915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:33.513688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42981 (local address 127.28.47.129:48317)
0811 22:09:33.514340 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:33.514393 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:33.515821 (+ 1428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:33.517035 (+ 1214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:33.517045 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:33.517538 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:33.518101 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:33.518117 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:33.519018 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:33.519021 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:33.520058 (+ 1037us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:33.520070 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:33.520664 (+ 594us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:33.520669 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:33.520866 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:33.521691 (+ 825us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:33.521719 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:33.523895 (+ 2176us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:33.526777 (+ 2882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:33.526787 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:33.526815 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:33.527241 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:33.527650 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:33.527653 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:33.527656 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:33.527856 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:33.528388 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:33.528399 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:33.528723 (+ 324us) client_negotiation.cc:769] Sending connection context
0811 22:09:33.528943 (+ 220us) client_negotiation.cc:240] Negotiation successful
0811 22:09:33.529403 (+ 460us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"thread_start_us":179,"threads_started":1}
I20260811 22:09:33.532114 912 heartbeater.cc:347] Connected to a master server at 127.28.47.190:42981
I20260811 22:09:33.532431 912 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:33.533042 912 heartbeater.cc:511] Master 127.28.47.190:42981 requested a full tablet report, sending...
I20260811 22:09:33.535238 681 ts_manager.cc:194] Registered new tserver with Master: 6a3e312d4cca4c948e3087127fc1dd2d (127.28.47.129:36549)
I20260811 22:09:33.536571 681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:48317
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:09:33.550688 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42981
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:33.676555 918 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:33.676927 918 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:33.677013 918 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:33.681600 918 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:33.681725 918 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:33.681826 918 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:33.688089 918 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.918
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:33.689391 918 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:33.690600 918 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:33.701099 927 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:33.701201 924 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:33.701117 925 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:33.701334 918 server_base.cc:1034] running on GCE node
I20260811 22:09:33.702227 918 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:33.703073 918 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:33.704284 918 hybrid_clock.cc:648] HybridClock initialized: now 1786486173704258 us; error 40 us; skew 500 ppm
Aug 11 22:09:33 dist-test-slave-vxj2 krb5kdc[630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486173, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:33.707644 918 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:09:33.708767 918 webserver.cc:460] Webserver started at http://127.28.47.130:42831/ using document root <none> and password file <none>
I20260811 22:09:33.709506 918 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:33.709606 918 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:33.709858 918 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:33.712132 918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "2b2b065f1be9413cb1c177684a3b9299"
format_stamp: "Formatted at 2026-08-11 22:09:33 on dist-test-slave-vxj2"
server_key: "fc5dbdcfe4b2dda0c5c2ed8687c0e6ed"
server_key_iv: "236b1b12613d75ac48b2048a36e4a214"
server_key_version: "encryptionkey@0"
I20260811 22:09:33.712775 918 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "2b2b065f1be9413cb1c177684a3b9299"
format_stamp: "Formatted at 2026-08-11 22:09:33 on dist-test-slave-vxj2"
server_key: "fc5dbdcfe4b2dda0c5c2ed8687c0e6ed"
server_key_iv: "236b1b12613d75ac48b2048a36e4a214"
server_key_version: "encryptionkey@0"
I20260811 22:09:33.716974 918 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:09:33.719789 934 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:33.721236 918 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:33.721383 918 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "2b2b065f1be9413cb1c177684a3b9299"
format_stamp: "Formatted at 2026-08-11 22:09:33 on dist-test-slave-vxj2"
server_key: "fc5dbdcfe4b2dda0c5c2ed8687c0e6ed"
server_key_iv: "236b1b12613d75ac48b2048a36e4a214"
server_key_version: "encryptionkey@0"
I20260811 22:09:33.721520 918 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:33.739073 918 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:33.743093 918 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:33.743333 918 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:33.744017 918 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:33.744953 918 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:33.745026 918 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:33.745100 918 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:33.745147 918 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:33.755303 918 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:40291
I20260811 22:09:33.755376 1068 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:40291 every 8 connection(s)
I20260811 22:09:33.756428 918 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:33.759670 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 918
I20260811 22:09:33.759824 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:09:33.760169 28862 external_mini_cluster.cc:1428] Setting key d67797e5ce98f78aefe8c7acadeaccc7
Aug 11 22:09:33 dist-test-slave-vxj2 krb5kdc[630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486173, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:09:33.773756 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:33.758611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:37615 (local address 127.28.47.190:42981)
0811 22:09:33.758784 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:33.758789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:33.759982 (+ 1193us) server_negotiation.cc:407] Connection header received
0811 22:09:33.762105 (+ 2123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:33.762108 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:33.762154 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:33.762215 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:33.764026 (+ 1811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:33.764964 (+ 938us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:33.766302 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:33.766591 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:33.766725 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:33.769688 (+ 2963us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:33.769697 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:33.769699 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:33.769730 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:33.771247 (+ 1517us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:33.771958 (+ 711us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:33.771962 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:33.771963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:33.772048 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:33.772434 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:33.772438 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:33.772440 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:33.772660 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:33.772771 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:33.773435 (+ 664us) server_negotiation.cc:299] Negotiation successful
0811 22:09:33.773581 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:09:33.774806 1071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:33.759117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42981 (local address 127.28.47.130:37615)
0811 22:09:33.759720 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:33.759843 (+ 123us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:33.761837 (+ 1994us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:33.762433 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:33.762445 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:33.763045 (+ 600us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:33.763851 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:33.763876 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:33.765133 (+ 1257us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:33.765136 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:33.766155 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:33.766166 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:33.766789 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:33.766793 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:33.767002 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:33.767751 (+ 749us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:33.767784 (+ 33us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:33.769458 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:33.771432 (+ 1974us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:33.771440 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:33.771460 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:33.771824 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:33.772146 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:33.772149 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:33.772151 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:33.772325 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:33.772774 (+ 449us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:33.772784 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:33.773097 (+ 313us) client_negotiation.cc:769] Sending connection context
0811 22:09:33.773364 (+ 267us) client_negotiation.cc:240] Negotiation successful
0811 22:09:33.773756 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":121,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
I20260811 22:09:33.776307 1069 heartbeater.cc:347] Connected to a master server at 127.28.47.190:42981
I20260811 22:09:33.776594 1069 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:33.777241 1069 heartbeater.cc:511] Master 127.28.47.190:42981 requested a full tablet report, sending...
I20260811 22:09:33.778491 681 ts_manager.cc:194] Registered new tserver with Master: 2b2b065f1be9413cb1c177684a3b9299 (127.28.47.130:40291)
I20260811 22:09:33.779397 681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:37615
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:09:33.796864 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42981
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:33.921351 1074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:33.921674 1074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:33.921761 1074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:33.926162 1074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:33.926242 1074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:33.926330 1074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:33.932211 1074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42981
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:33.933568 1074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:33.934894 1074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:33.945526 1082 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:33.945555 1084 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:33.945777 1080 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:33.945791 1074 server_base.cc:1034] running on GCE node
I20260811 22:09:33.946501 1074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:33.947278 1074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:33.948486 1074 hybrid_clock.cc:648] HybridClock initialized: now 1786486173948466 us; error 34 us; skew 500 ppm
Aug 11 22:09:33 dist-test-slave-vxj2 krb5kdc[630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486173, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:33.951694 1074 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:09:33.952769 1074 webserver.cc:460] Webserver started at http://127.28.47.131:42757/ using document root <none> and password file <none>
I20260811 22:09:33.953467 1074 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:33.953564 1074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:33.953809 1074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:33.956089 1074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "72c2362facdf4665a8e5e236446ff0cc"
format_stamp: "Formatted at 2026-08-11 22:09:33 on dist-test-slave-vxj2"
server_key: "f14ea8f752bc0d1959c5a2b734cf3de3"
server_key_iv: "9b6b96233033bc06d030adec79cb4212"
server_key_version: "encryptionkey@0"
I20260811 22:09:33.956817 1074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "72c2362facdf4665a8e5e236446ff0cc"
format_stamp: "Formatted at 2026-08-11 22:09:33 on dist-test-slave-vxj2"
server_key: "f14ea8f752bc0d1959c5a2b734cf3de3"
server_key_iv: "9b6b96233033bc06d030adec79cb4212"
server_key_version: "encryptionkey@0"
I20260811 22:09:33.961318 1074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:09:33.964131 1093 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:33.965806 1074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:33.965963 1074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "72c2362facdf4665a8e5e236446ff0cc"
format_stamp: "Formatted at 2026-08-11 22:09:33 on dist-test-slave-vxj2"
server_key: "f14ea8f752bc0d1959c5a2b734cf3de3"
server_key_iv: "9b6b96233033bc06d030adec79cb4212"
server_key_version: "encryptionkey@0"
I20260811 22:09:33.966094 1074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:33.984549 1074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:33.988600 1074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:33.988838 1074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:33.989566 1074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:33.990492 1074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:33.990566 1074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:33.990638 1074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:33.990685 1074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:34.000763 1074 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:33609
I20260811 22:09:34.000828 1225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:33609 every 8 connection(s)
I20260811 22:09:34.002131 1074 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:34.005349 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 1074
I20260811 22:09:34.005497 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:09:34.005805 28862 external_mini_cluster.cc:1428] Setting key db6482dd7896273373ef889d1ee517c9
Aug 11 22:09:34 dist-test-slave-vxj2 krb5kdc[630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486173, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:34.017722 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:34.004189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:44565 (local address 127.28.47.190:42981)
0811 22:09:34.004337 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:34.004342 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:34.005355 (+ 1013us) server_negotiation.cc:407] Connection header received
0811 22:09:34.007140 (+ 1785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:34.007143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:34.007203 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:34.007302 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:34.008665 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:34.009301 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:34.010574 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:34.010813 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:34.010935 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:34.013810 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:34.013815 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:34.013817 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:34.013852 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:34.015408 (+ 1556us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:34.015997 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:34.016001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:34.016003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:34.016092 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:34.016456 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:34.016460 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:34.016463 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:34.016671 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:34.016782 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:34.017394 (+ 612us) server_negotiation.cc:299] Negotiation successful
0811 22:09:34.017548 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:09:34.018821 1228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:34.004618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42981 (local address 127.28.47.131:44565)
0811 22:09:34.005136 (+ 518us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:34.005235 (+ 99us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:34.006919 (+ 1684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:34.007497 (+ 578us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:34.007505 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:34.007890 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:34.008517 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:34.008532 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:34.009443 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:34.009446 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:34.010416 (+ 970us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:34.010427 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:34.010991 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:34.010995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:34.011198 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:34.011896 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:34.011917 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:34.013588 (+ 1671us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:34.015556 (+ 1968us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:34.015563 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:34.015583 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:34.015886 (+ 303us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:34.016188 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:34.016191 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:34.016193 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:34.016359 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:34.016782 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:34.016791 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:34.017085 (+ 294us) client_negotiation.cc:769] Sending connection context
0811 22:09:34.017350 (+ 265us) client_negotiation.cc:240] Negotiation successful
0811 22:09:34.017739 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":126,"threads_started":1}
I20260811 22:09:34.020217 1226 heartbeater.cc:347] Connected to a master server at 127.28.47.190:42981
I20260811 22:09:34.020547 1226 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:34.021127 1226 heartbeater.cc:511] Master 127.28.47.190:42981 requested a full tablet report, sending...
I20260811 22:09:34.022315 681 ts_manager.cc:194] Registered new tserver with Master: 72c2362facdf4665a8e5e236446ff0cc (127.28.47.131:33609)
I20260811 22:09:34.022996 681 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:44565
I20260811 22:09:34.033052 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:09:34 dist-test-slave-vxj2 krb5kdc[630](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:09:34 dist-test-slave-vxj2 krb5kdc[630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.129@KRBTEST.COM
I20260811 22:09:34.062598 1233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:34.051961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35096 (local address 127.28.47.129:36549)
0811 22:09:34.052257 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:34.052265 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:34.052296 (+ 31us) server_negotiation.cc:407] Connection header received
0811 22:09:34.052432 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:34.052438 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:34.052642 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:34.052821 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:34.053629 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:34.054399 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:34.055388 (+ 989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:34.055794 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:34.055913 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:34.058628 (+ 2715us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:34.058641 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:34.058657 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:34.058687 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:34.060523 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:34.061024 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:34.061030 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:34.061031 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:34.061121 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:34.061454 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:34.061458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:34.061460 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:34.061747 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:34.061984 (+ 237us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:34.062229 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 22:09:34.062417 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":69,"threads_started":1}
Aug 11 22:09:34 dist-test-slave-vxj2 krb5kdc[630](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486174, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:34.075688 728 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:34.065993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58778 (local address 127.28.47.190:42981)
0811 22:09:34.066161 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:34.066165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:34.066181 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:09:34.066241 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:34.066244 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:34.066301 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:34.066376 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:34.067047 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:34.067642 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:34.068467 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:34.068707 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:34.068802 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:34.071720 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:34.071723 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:34.071726 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:34.071763 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:34.073516 (+ 1753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:34.074059 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:34.074062 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:34.074064 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:34.074160 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:34.074527 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:34.074530 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:34.074532 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:34.074898 (+ 366us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:34.075039 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:34.075292 (+ 253us) server_negotiation.cc:299] Negotiation successful
0811 22:09:34.075422 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
W20260811 22:09:34.076248 681 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58778
I20260811 22:09:34.076959 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 755
I20260811 22:09:34.094193 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 918
I20260811 22:09:34.114711 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 1074
I20260811 22:09:34.124367 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 647
2026-08-11T22:09:34Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2495 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:34 dist-test-slave-vxj2 krb5kdc[1238](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:34 dist-test-slave-vxj2 krb5kdc[1238](info): set up 2 sockets
Aug 11 22:09:34 dist-test-slave-vxj2 krb5kdc[1238](info): commencing operation
krb5kdc: starting...
W20260811 22:09:35.669113 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.519s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:09:35 dist-test-slave-vxj2 krb5kdc[1238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:09:35.882371 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:35843
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:35843
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:36.133560 1252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:36.134267 1252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:36.134449 1252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:36.142887 1252 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:36.143066 1252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:36.143157 1252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:36.143255 1252 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:36.143366 1252 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:36.153710 1252 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:35843
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:35843
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:36.155766 1252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:36.158017 1252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:36.173584 1259 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:36.173713 1263 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:36.173651 1258 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:36.174304 1252 server_base.cc:1034] running on GCE node
I20260811 22:09:36.175549 1252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:36.178676 1252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:36.180346 1252 hybrid_clock.cc:648] HybridClock initialized: now 1786486176180335 us; error 106 us; skew 500 ppm
Aug 11 22:09:36 dist-test-slave-vxj2 krb5kdc[1238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486176, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:36.186721 1252 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:09:36.188428 1252 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:36.188580 1252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:36.188894 1252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:36.193089 1252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "caa3dbb32ae843259f3ff8baa0bd1173"
format_stamp: "Formatted at 2026-08-11 22:09:36 on dist-test-slave-vxj2"
server_key: "aa0ac5c1cf603df4800f70e7e1ab0185"
server_key_iv: "e333f047604fe6d7096526992fc91272"
server_key_version: "encryptionkey@0"
I20260811 22:09:36.194494 1252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "caa3dbb32ae843259f3ff8baa0bd1173"
format_stamp: "Formatted at 2026-08-11 22:09:36 on dist-test-slave-vxj2"
server_key: "aa0ac5c1cf603df4800f70e7e1ab0185"
server_key_iv: "e333f047604fe6d7096526992fc91272"
server_key_version: "encryptionkey@0"
I20260811 22:09:36.202756 1252 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.001s
I20260811 22:09:36.207962 1271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:36.210883 1252 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 22:09:36.211131 1252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "caa3dbb32ae843259f3ff8baa0bd1173"
format_stamp: "Formatted at 2026-08-11 22:09:36 on dist-test-slave-vxj2"
server_key: "aa0ac5c1cf603df4800f70e7e1ab0185"
server_key_iv: "e333f047604fe6d7096526992fc91272"
server_key_version: "encryptionkey@0"
I20260811 22:09:36.211354 1252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:36.240633 1252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:36.247159 1252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:36.247464 1252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:36.262807 1252 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:35843
I20260811 22:09:36.263314 1330 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:35843 every 8 connection(s)
I20260811 22:09:36.264150 1252 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:36.269538 1331 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:36.270504 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 1252
I20260811 22:09:36.270759 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:36.271209 28862 external_mini_cluster.cc:1428] Setting key 8020efebe54a17deaa255acdcb812baf
I20260811 22:09:36.280988 1331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173: Bootstrap starting.
I20260811 22:09:36.285337 1331 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:36.286926 1331 log.cc:824] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:09:36 dist-test-slave-vxj2 krb5kdc[1238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:36.290750 1331 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173: No bootstrap required, opened a new log
I20260811 22:09:36.298249 1331 raft_consensus.cc:348] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa3dbb32ae843259f3ff8baa0bd1173" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35843 } }
I20260811 22:09:36.298719 1331 raft_consensus.cc:374] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:36.298834 1331 raft_consensus.cc:729] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: caa3dbb32ae843259f3ff8baa0bd1173, State: Initialized, Role: FOLLOWER
I20260811 22:09:36.299630 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:36.273930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51638 (local address 127.28.47.190:35843)
0811 22:09:36.274922 (+ 992us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:36.274942 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:36.275007 (+ 65us) server_negotiation.cc:407] Connection header received
0811 22:09:36.275977 (+ 970us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:36.276018 (+ 41us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:36.276704 (+ 686us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:36.277443 (+ 739us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:36.279090 (+ 1647us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:36.280754 (+ 1664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:36.282394 (+ 1640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:36.283318 (+ 924us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:36.283688 (+ 370us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:36.288647 (+ 4959us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:36.288664 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:36.288692 (+ 28us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:36.288739 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:36.292168 (+ 3429us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:36.293095 (+ 927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:36.293111 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:36.293117 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:36.293426 (+ 309us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:36.294204 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:36.294208 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:36.294209 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:36.295209 (+ 1000us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:36.295533 (+ 324us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:36.297339 (+ 1806us) server_negotiation.cc:299] Negotiation successful
0811 22:09:36.297961 (+ 622us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":666,"thread_start_us":309,"threads_started":1}
I20260811 22:09:36.299633 1331 consensus_queue.cc:260] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa3dbb32ae843259f3ff8baa0bd1173" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35843 } }
I20260811 22:09:36.300064 1331 raft_consensus.cc:388] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:36.300164 1331 raft_consensus.cc:482] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:36.300290 1331 raft_consensus.cc:3037] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:36.302130 1331 raft_consensus.cc:504] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa3dbb32ae843259f3ff8baa0bd1173" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35843 } }
I20260811 22:09:36.302713 1331 leader_election.cc:302] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: caa3dbb32ae843259f3ff8baa0bd1173; no voters:
I20260811 22:09:36.303313 1331 leader_election.cc:288] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:36.303440 1336 raft_consensus.cc:2781] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:36.304136 1336 raft_consensus.cc:686] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [term 1 LEADER]: Becoming Leader. State: Replica: caa3dbb32ae843259f3ff8baa0bd1173, State: Running, Role: LEADER
I20260811 22:09:36.304711 1336 consensus_queue.cc:237] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa3dbb32ae843259f3ff8baa0bd1173" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35843 } }
I20260811 22:09:36.305969 1331 sys_catalog.cc:564] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:36.308213 1337 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "caa3dbb32ae843259f3ff8baa0bd1173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa3dbb32ae843259f3ff8baa0bd1173" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35843 } } }
I20260811 22:09:36.308173 1338 sys_catalog.cc:455] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [sys.catalog]: SysCatalogTable state changed. Reason: New leader caa3dbb32ae843259f3ff8baa0bd1173. Latest consensus state: current_term: 1 leader_uuid: "caa3dbb32ae843259f3ff8baa0bd1173" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "caa3dbb32ae843259f3ff8baa0bd1173" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35843 } } }
I20260811 22:09:36.308494 1337 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:36.308533 1338 sys_catalog.cc:458] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:36.309333 1348 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:36.316550 1348 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:36.327998 1348 catalog_manager.cc:1347] Generated new cluster ID: 7fef2cc8b73a45f1adc4f934225fc127
I20260811 22:09:36.328142 1348 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:36.354874 1348 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:36.356815 1348 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:36.371728 1348 catalog_manager.cc:5868] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173: Generated new TSK 0
I20260811 22:09:36.373234 1348 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:09:36.423552 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35843
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:36.633555 1360 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:36.634104 1360 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:36.634250 1360 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:36.640852 1360 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:36.641002 1360 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:36.641188 1360 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:36.650501 1360 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1360
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:36.652585 1360 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:36.654385 1360 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:36.671926 1369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:36.671892 1372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:36.671892 1368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:36.672595 1360 server_base.cc:1034] running on GCE node
I20260811 22:09:36.673578 1360 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:36.674885 1360 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:36.676330 1360 hybrid_clock.cc:648] HybridClock initialized: now 1786486176676295 us; error 68 us; skew 500 ppm
Aug 11 22:09:36 dist-test-slave-vxj2 krb5kdc[1238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486176, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:36.682145 1360 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:09:36.683533 1360 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:36.683704 1360 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:36.684094 1360 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:36.687920 1360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "e2a6d0152e22481882c084207b9a2fe4"
format_stamp: "Formatted at 2026-08-11 22:09:36 on dist-test-slave-vxj2"
server_key: "d965fb31ee79ef391fa95e185f954aaa"
server_key_iv: "233087d2d31acb90a0684c0b3ceec82a"
server_key_version: "encryptionkey@0"
I20260811 22:09:36.689026 1360 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "e2a6d0152e22481882c084207b9a2fe4"
format_stamp: "Formatted at 2026-08-11 22:09:36 on dist-test-slave-vxj2"
server_key: "d965fb31ee79ef391fa95e185f954aaa"
server_key_iv: "233087d2d31acb90a0684c0b3ceec82a"
server_key_version: "encryptionkey@0"
I20260811 22:09:36.696580 1360 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.005s sys 0.000s
I20260811 22:09:36.701541 1378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:36.704259 1360 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.001s sys 0.003s
I20260811 22:09:36.704629 1360 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "e2a6d0152e22481882c084207b9a2fe4"
format_stamp: "Formatted at 2026-08-11 22:09:36 on dist-test-slave-vxj2"
server_key: "d965fb31ee79ef391fa95e185f954aaa"
server_key_iv: "233087d2d31acb90a0684c0b3ceec82a"
server_key_version: "encryptionkey@0"
I20260811 22:09:36.704900 1360 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:36.752900 1360 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:36.758968 1360 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:36.759337 1360 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:36.760407 1360 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:36.762061 1360 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:36.762171 1360 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:36.762311 1360 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:36.762387 1360 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:36.779568 1360 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:34219
I20260811 22:09:36.779609 1513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:34219 every 8 connection(s)
I20260811 22:09:36.780782 1360 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:36.789963 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 1360
I20260811 22:09:36.790217 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:09:36.790861 28862 external_mini_cluster.cc:1428] Setting key f34fd11bc453c5133583743275bf6080
Aug 11 22:09:36 dist-test-slave-vxj2 krb5kdc[1238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486176, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
I20260811 22:09:36.813123 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:36.784420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:50697 (local address 127.28.47.190:35843)
0811 22:09:36.784852 (+ 432us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:36.784857 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:36.786589 (+ 1732us) server_negotiation.cc:407] Connection header received
0811 22:09:36.789315 (+ 2726us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:36.789326 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:36.789423 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:36.789569 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:36.792887 (+ 3318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:36.794046 (+ 1159us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:36.798028 (+ 3982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:36.798426 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:36.798676 (+ 250us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:36.804671 (+ 5995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:36.804681 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:36.804698 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:36.804767 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:36.808372 (+ 3605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:36.809742 (+ 1370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:36.809759 (+ 17us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:36.809761 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:36.809956 (+ 195us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:36.810742 (+ 786us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:36.810751 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:36.810756 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:36.811274 (+ 518us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:36.811462 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:36.812637 (+ 1175us) server_negotiation.cc:299] Negotiation successful
0811 22:09:36.812852 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 22:09:36.814760 1516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:36.785272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:35843 (local address 127.28.47.129:50697)
0811 22:09:36.786276 (+ 1004us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:36.786377 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:36.788753 (+ 2376us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:36.790143 (+ 1390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:36.790171 (+ 28us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:36.791327 (+ 1156us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:36.792643 (+ 1316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:36.792676 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:36.794304 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:36.794311 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:36.797636 (+ 3325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:36.797684 (+ 48us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:36.798816 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:36.798828 (+ 12us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:36.799161 (+ 333us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:36.800380 (+ 1219us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:36.800431 (+ 51us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:36.804267 (+ 3836us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:36.808676 (+ 4409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:36.808755 (+ 79us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:36.808789 (+ 34us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:36.809540 (+ 751us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:36.810172 (+ 632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:36.810178 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:36.810182 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:36.810546 (+ 364us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:36.811488 (+ 942us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:36.811496 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:36.812164 (+ 668us) client_negotiation.cc:769] Sending connection context
0811 22:09:36.812484 (+ 320us) client_negotiation.cc:240] Negotiation successful
0811 22:09:36.813001 (+ 517us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":246,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:09:36.817510 1514 heartbeater.cc:347] Connected to a master server at 127.28.47.190:35843
I20260811 22:09:36.817950 1514 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:36.818910 1514 heartbeater.cc:511] Master 127.28.47.190:35843 requested a full tablet report, sending...
I20260811 22:09:36.822175 1292 ts_manager.cc:194] Registered new tserver with Master: e2a6d0152e22481882c084207b9a2fe4 (127.28.47.129:34219)
I20260811 22:09:36.824095 1292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:50697
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:09:36.842236 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35843
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:37.049618 1519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:37.050145 1519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:37.050225 1519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:37.056875 1519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:37.057014 1519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:37.057124 1519 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:37.066197 1519 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:37.068094 1519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:37.069897 1519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:37.086939 1529 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:37.086971 1525 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:37.086974 1526 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:37.088213 1519 server_base.cc:1034] running on GCE node
I20260811 22:09:37.089107 1519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:37.090461 1519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:37.093852 1519 hybrid_clock.cc:648] HybridClock initialized: now 1786486177093772 us; error 96 us; skew 500 ppm
Aug 11 22:09:37 dist-test-slave-vxj2 krb5kdc[1238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486177, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:37.099772 1519 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:09:37.101310 1519 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:37.101442 1519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:37.101893 1519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:37.105700 1519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "862bebb847584850aa16b87c6ebd43f5"
format_stamp: "Formatted at 2026-08-11 22:09:37 on dist-test-slave-vxj2"
server_key: "e8fe0bd81e76d3937fa65dea97c6aa2b"
server_key_iv: "5b6ddb727f549e25b9ace0dbea5faa20"
server_key_version: "encryptionkey@0"
I20260811 22:09:37.107020 1519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "862bebb847584850aa16b87c6ebd43f5"
format_stamp: "Formatted at 2026-08-11 22:09:37 on dist-test-slave-vxj2"
server_key: "e8fe0bd81e76d3937fa65dea97c6aa2b"
server_key_iv: "5b6ddb727f549e25b9ace0dbea5faa20"
server_key_version: "encryptionkey@0"
I20260811 22:09:37.114475 1519 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.001s
I20260811 22:09:37.118906 1537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:37.122085 1519 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:09:37.122390 1519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "862bebb847584850aa16b87c6ebd43f5"
format_stamp: "Formatted at 2026-08-11 22:09:37 on dist-test-slave-vxj2"
server_key: "e8fe0bd81e76d3937fa65dea97c6aa2b"
server_key_iv: "5b6ddb727f549e25b9ace0dbea5faa20"
server_key_version: "encryptionkey@0"
I20260811 22:09:37.122644 1519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:37.146337 1519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:37.152372 1519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:37.152667 1519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:37.153656 1519 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:37.155121 1519 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:37.155203 1519 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:37.155274 1519 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:37.155328 1519 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:37.172648 1519 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:33787
I20260811 22:09:37.172681 1670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:33787 every 8 connection(s)
I20260811 22:09:37.173802 1519 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:37.177759 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 1519
I20260811 22:09:37.178016 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:09:37.178675 28862 external_mini_cluster.cc:1428] Setting key c2d421f2345cf9b9558c77c0bdec8001
Aug 11 22:09:37 dist-test-slave-vxj2 krb5kdc[1238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486177, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
I20260811 22:09:37.202868 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.176863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:37951 (local address 127.28.47.190:35843)
0811 22:09:37.177112 (+ 249us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:37.177124 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:37.179147 (+ 2023us) server_negotiation.cc:407] Connection header received
0811 22:09:37.182040 (+ 2893us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:37.182044 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:37.182122 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:37.182225 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:37.185628 (+ 3403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.186408 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.188567 (+ 2159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.188929 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.189104 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:37.195489 (+ 6385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:37.195501 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:37.195503 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:37.195555 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:37.198285 (+ 2730us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:37.199701 (+ 1416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:37.199715 (+ 14us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:37.199717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:37.199886 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:37.200611 (+ 725us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:37.200614 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:37.200615 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:37.201015 (+ 400us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:37.201231 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:37.202369 (+ 1138us) server_negotiation.cc:299] Negotiation successful
0811 22:09:37.202647 (+ 278us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":97}
I20260811 22:09:37.204680 1673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.177705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:35843 (local address 127.28.47.130:37951)
0811 22:09:37.178819 (+ 1114us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:37.178945 (+ 126us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:37.181587 (+ 2642us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:37.182571 (+ 984us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:37.182586 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:37.183719 (+ 1133us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:37.185277 (+ 1558us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:37.185332 (+ 55us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.186695 (+ 1363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.186698 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:37.188228 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:37.188247 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.189274 (+ 1027us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.189282 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:37.189790 (+ 508us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:37.191061 (+ 1271us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:37.191117 (+ 56us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:37.195066 (+ 3949us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:37.198537 (+ 3471us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:37.198546 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:37.198579 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:37.199454 (+ 875us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:37.200020 (+ 566us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:37.200022 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:37.200023 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:37.200379 (+ 356us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:37.201236 (+ 857us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:37.201252 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:37.201797 (+ 545us) client_negotiation.cc:769] Sending connection context
0811 22:09:37.202227 (+ 430us) client_negotiation.cc:240] Negotiation successful
0811 22:09:37.202845 (+ 618us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"thread_start_us":298,"threads_started":1}
I20260811 22:09:37.206781 1671 heartbeater.cc:347] Connected to a master server at 127.28.47.190:35843
I20260811 22:09:37.207283 1671 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:37.208244 1671 heartbeater.cc:511] Master 127.28.47.190:35843 requested a full tablet report, sending...
I20260811 22:09:37.210256 1292 ts_manager.cc:194] Registered new tserver with Master: 862bebb847584850aa16b87c6ebd43f5 (127.28.47.130:33787)
I20260811 22:09:37.211838 1292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:37951
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:09:37.231058 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35843
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:40113
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:37.446491 1676 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:37.446990 1676 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:37.447068 1676 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:37.453470 1676 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:37.453599 1676 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:37.453709 1676 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:37.462946 1676 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:40113
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35843
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1676
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:37.465260 1676 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:37.467278 1676 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:37.484617 1682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:37.484617 1686 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:37.485327 1676 server_base.cc:1034] running on GCE node
W20260811 22:09:37.484617 1683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:37.486176 1676 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:37.487504 1676 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:37.490952 1676 hybrid_clock.cc:648] HybridClock initialized: now 1786486177490853 us; error 103 us; skew 500 ppm
Aug 11 22:09:37 dist-test-slave-vxj2 krb5kdc[1238](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486177, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:37.496795 1676 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:09:37.498610 1676 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:37.498741 1676 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:37.499037 1676 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:37.503865 1676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "b45cf06068124b8bbbdfce7bd20af55a"
format_stamp: "Formatted at 2026-08-11 22:09:37 on dist-test-slave-vxj2"
server_key: "f23a776dba0ef427c7cb652f7fc0d4e4"
server_key_iv: "9b9440069dab0f5a7dff2fc1b9c1017f"
server_key_version: "encryptionkey@0"
I20260811 22:09:37.505137 1676 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "b45cf06068124b8bbbdfce7bd20af55a"
format_stamp: "Formatted at 2026-08-11 22:09:37 on dist-test-slave-vxj2"
server_key: "f23a776dba0ef427c7cb652f7fc0d4e4"
server_key_iv: "9b9440069dab0f5a7dff2fc1b9c1017f"
server_key_version: "encryptionkey@0"
I20260811 22:09:37.513883 1676 fs_manager.cc:691] Time spent creating directory manager: real 0.008s user 0.007s sys 0.000s
I20260811 22:09:37.518565 1692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:37.521384 1676 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:09:37.521626 1676 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "b45cf06068124b8bbbdfce7bd20af55a"
format_stamp: "Formatted at 2026-08-11 22:09:37 on dist-test-slave-vxj2"
server_key: "f23a776dba0ef427c7cb652f7fc0d4e4"
server_key_iv: "9b9440069dab0f5a7dff2fc1b9c1017f"
server_key_version: "encryptionkey@0"
I20260811 22:09:37.521839 1676 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:37.543143 1676 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:37.549998 1676 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:37.550384 1676 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:37.551579 1676 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:37.553269 1676 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:37.553341 1676 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:37.553457 1676 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:37.553511 1676 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:37.571808 1676 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:38769
I20260811 22:09:37.571838 1821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:38769 every 8 connection(s)
I20260811 22:09:37.573040 1676 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:37.576781 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 1676
I20260811 22:09:37.577128 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:09:37.577769 28862 external_mini_cluster.cc:1428] Setting key d8105d479024de0dede14f0555eafece
Aug 11 22:09:37 dist-test-slave-vxj2 krb5kdc[1238](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486177, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:37.602986 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.576648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:60089 (local address 127.28.47.190:35843)
0811 22:09:37.576946 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:37.576952 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:37.579452 (+ 2500us) server_negotiation.cc:407] Connection header received
0811 22:09:37.582723 (+ 3271us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:37.582728 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:37.582835 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:37.582925 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:37.586120 (+ 3195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.587178 (+ 1058us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.589707 (+ 2529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.590095 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.590252 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:37.595677 (+ 5425us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:37.595690 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:37.595692 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:37.595743 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:37.598799 (+ 3056us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:37.599944 (+ 1145us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:37.599948 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:37.599949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:37.600103 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:37.600806 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:37.600810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:37.600811 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:37.601270 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:37.601419 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:37.602504 (+ 1085us) server_negotiation.cc:299] Negotiation successful
0811 22:09:37.602785 (+ 281us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260811 22:09:37.604624 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.577921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:35843 (local address 127.28.47.131:60089)
0811 22:09:37.579065 (+ 1144us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:37.579189 (+ 124us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:37.582376 (+ 3187us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:37.583239 (+ 863us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:37.583249 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:37.584138 (+ 889us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:37.585761 (+ 1623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:37.585814 (+ 53us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.587420 (+ 1606us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.587424 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:37.589479 (+ 2055us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:37.589494 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.590332 (+ 838us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.590336 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:37.590622 (+ 286us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:37.591613 (+ 991us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:37.591644 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:37.595297 (+ 3653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:37.599031 (+ 3734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:37.599047 (+ 16us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:37.599075 (+ 28us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:37.599670 (+ 595us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:37.600283 (+ 613us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:37.600286 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:37.600288 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:37.600597 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:37.601478 (+ 881us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:37.601485 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:37.602029 (+ 544us) client_negotiation.cc:769] Sending connection context
0811 22:09:37.602355 (+ 326us) client_negotiation.cc:240] Negotiation successful
0811 22:09:37.602949 (+ 594us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":756,"thread_start_us":341,"threads_started":1}
I20260811 22:09:37.606910 1822 heartbeater.cc:347] Connected to a master server at 127.28.47.190:35843
I20260811 22:09:37.607592 1822 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:37.608631 1822 heartbeater.cc:511] Master 127.28.47.190:35843 requested a full tablet report, sending...
I20260811 22:09:37.610345 1292 ts_manager.cc:194] Registered new tserver with Master: b45cf06068124b8bbbdfce7bd20af55a (127.28.47.131:38769)
I20260811 22:09:37.611285 1292 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:60089
I20260811 22:09:37.622284 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:09:37.639703 1334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.625609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51650 (local address 127.28.47.190:35843)
0811 22:09:37.625877 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:37.625887 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:37.626351 (+ 464us) server_negotiation.cc:407] Connection header received
0811 22:09:37.626544 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:37.626547 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:37.626622 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:37.626821 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:37.628106 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.629271 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.630662 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.631103 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.631344 (+ 241us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:37.633818 (+ 2474us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:37.633823 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:37.633825 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:37.633868 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:37.636285 (+ 2417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:37.637301 (+ 1016us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:37.637305 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:37.637306 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:37.637515 (+ 209us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:37.637989 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:37.637992 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:37.637993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:37.638395 (+ 402us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:37.638588 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:37.639158 (+ 570us) server_negotiation.cc:299] Negotiation successful
0811 22:09:37.639339 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:09:37.645471 1292 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:09:37.650940 1292 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:37.680341 1837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.668371 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:33787 (local address 127.0.0.1:46154)
0811 22:09:37.669269 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:37.669379 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:37.669919 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:37.671446 (+ 1527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:37.671454 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:37.671512 (+ 58us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:37.671797 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:37.671814 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.674009 (+ 2195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.674024 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:37.676516 (+ 2492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:37.676528 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.679067 (+ 2539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.679077 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:37.679307 (+ 230us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:37.679385 (+ 78us) client_negotiation.cc:769] Sending connection context
0811 22:09:37.679616 (+ 231us) client_negotiation.cc:240] Negotiation successful
0811 22:09:37.679896 (+ 280us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":484,"mutex_wait_us":53,"thread_start_us":319,"threads_started":1}
I20260811 22:09:37.681746 1838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.668510 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46154 (local address 127.28.47.130:33787)
0811 22:09:37.669472 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:37.669484 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:37.669535 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:09:37.670436 (+ 901us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:37.670454 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:37.670907 (+ 453us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:37.671303 (+ 396us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:37.672017 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.673660 (+ 1643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.676761 (+ 3101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.678268 (+ 1507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.678612 (+ 344us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:37.678924 (+ 312us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:37.681173 (+ 2249us) server_negotiation.cc:299] Negotiation successful
0811 22:09:37.681436 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":203,"threads_started":1}
I20260811 22:09:37.681861 1839 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.669422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:34219 (local address 127.0.0.1:43568)
0811 22:09:37.669860 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:37.669874 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:37.670031 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:37.672857 (+ 2826us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:37.672870 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:37.672894 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:37.673201 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:37.673220 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.675596 (+ 2376us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.675599 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:37.678638 (+ 3039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:37.678654 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.680994 (+ 2340us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.681009 (+ 15us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:37.681236 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:37.681304 (+ 68us) client_negotiation.cc:769] Sending connection context
0811 22:09:37.681412 (+ 108us) client_negotiation.cc:240] Negotiation successful
0811 22:09:37.681582 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"mutex_wait_us":431,"thread_start_us":181,"threads_started":1}
I20260811 22:09:37.682684 1840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.669868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:38769 (local address 127.0.0.1:44016)
0811 22:09:37.670805 (+ 937us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:37.670833 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:37.670979 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:37.673340 (+ 2361us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:37.673345 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:37.673380 (+ 35us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:37.673567 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:37.673587 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.675566 (+ 1979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.675569 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:37.678639 (+ 3070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:37.678654 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.681369 (+ 2715us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.681385 (+ 16us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:37.681935 (+ 550us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:37.681995 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 22:09:37.682172 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 22:09:37.682413 (+ 241us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":836,"mutex_wait_us":611,"thread_start_us":139,"threads_started":1}
I20260811 22:09:37.683499 1841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.670164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43568 (local address 127.28.47.129:34219)
0811 22:09:37.671865 (+ 1701us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:37.671886 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:37.671927 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:09:37.672201 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:37.672210 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:37.672609 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:37.672883 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:37.673404 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.675441 (+ 2037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.678942 (+ 3501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.680580 (+ 1638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.682486 (+ 1906us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:37.682825 (+ 339us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:37.682946 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 22:09:37.683189 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1526,"thread_start_us":224,"threads_started":1}
I20260811 22:09:37.683506 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.670946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44016 (local address 127.28.47.131:38769)
0811 22:09:37.671861 (+ 915us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:37.671888 (+ 27us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:37.671927 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:09:37.672201 (+ 274us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:37.672210 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:37.672609 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:37.672883 (+ 274us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:37.673797 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.675385 (+ 1588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.678902 (+ 3517us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:37.681115 (+ 2213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:37.681381 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:37.681677 (+ 296us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:37.682975 (+ 1298us) server_negotiation.cc:299] Negotiation successful
0811 22:09:37.683266 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":742,"thread_start_us":377,"threads_started":1}
I20260811 22:09:37.688841 1435 tablet_service.cc:1467] Processing CreateTablet for tablet 6a796f0b4e184511a0d0d8e157a6c24f (DEFAULT_TABLE table=test-table [id=5a1ee5e80dce41b3a9a96321941988d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:37.689005 1596 tablet_service.cc:1467] Processing CreateTablet for tablet 6a796f0b4e184511a0d0d8e157a6c24f (DEFAULT_TABLE table=test-table [id=5a1ee5e80dce41b3a9a96321941988d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:37.689612 1747 tablet_service.cc:1467] Processing CreateTablet for tablet 6a796f0b4e184511a0d0d8e157a6c24f (DEFAULT_TABLE table=test-table [id=5a1ee5e80dce41b3a9a96321941988d9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:37.691037 1435 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a796f0b4e184511a0d0d8e157a6c24f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:37.691403 1596 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a796f0b4e184511a0d0d8e157a6c24f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:37.691403 1747 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6a796f0b4e184511a0d0d8e157a6c24f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:37.704566 1843 tablet_bootstrap.cc:492] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4: Bootstrap starting.
I20260811 22:09:37.708588 1845 tablet_bootstrap.cc:492] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5: Bootstrap starting.
I20260811 22:09:37.709919 1843 tablet_bootstrap.cc:654] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:37.709784 1844 tablet_bootstrap.cc:492] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a: Bootstrap starting.
I20260811 22:09:37.713009 1843 log.cc:824] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:37.714434 1845 tablet_bootstrap.cc:654] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:37.715974 1844 tablet_bootstrap.cc:654] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:37.717027 1843 tablet_bootstrap.cc:492] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4: No bootstrap required, opened a new log
I20260811 22:09:37.716920 1845 log.cc:824] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:37.717561 1843 ts_tablet_manager.cc:1397] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4: Time spent bootstrapping tablet: real 0.013s user 0.012s sys 0.000s
I20260811 22:09:37.718911 1844 log.cc:824] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:37.722060 1845 tablet_bootstrap.cc:492] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5: No bootstrap required, opened a new log
I20260811 22:09:37.722635 1845 ts_tablet_manager.cc:1397] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5: Time spent bootstrapping tablet: real 0.014s user 0.012s sys 0.000s
I20260811 22:09:37.723955 1844 tablet_bootstrap.cc:492] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a: No bootstrap required, opened a new log
I20260811 22:09:37.724306 1844 ts_tablet_manager.cc:1397] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a: Time spent bootstrapping tablet: real 0.015s user 0.013s sys 0.000s
I20260811 22:09:37.726861 1843 raft_consensus.cc:348] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "862bebb847584850aa16b87c6ebd43f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33787 } } peers { permanent_uuid: "b45cf06068124b8bbbdfce7bd20af55a" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38769 } } peers { permanent_uuid: "e2a6d0152e22481882c084207b9a2fe4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34219 } }
I20260811 22:09:37.727543 1843 raft_consensus.cc:374] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:37.727669 1843 raft_consensus.cc:729] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2a6d0152e22481882c084207b9a2fe4, State: Initialized, Role: FOLLOWER
I20260811 22:09:37.728950 1843 consensus_queue.cc:260] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "862bebb847584850aa16b87c6ebd43f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33787 } } peers { permanent_uuid: "b45cf06068124b8bbbdfce7bd20af55a" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38769 } } peers { permanent_uuid: "e2a6d0152e22481882c084207b9a2fe4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34219 } }
I20260811 22:09:37.730937 1514 heartbeater.cc:503] Master 127.28.47.190:35843 was elected leader, sending a full tablet report...
I20260811 22:09:37.731611 1843 ts_tablet_manager.cc:1428] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4: Time spent starting tablet: real 0.014s user 0.014s sys 0.000s
I20260811 22:09:37.733364 1844 raft_consensus.cc:348] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "862bebb847584850aa16b87c6ebd43f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33787 } } peers { permanent_uuid: "b45cf06068124b8bbbdfce7bd20af55a" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38769 } } peers { permanent_uuid: "e2a6d0152e22481882c084207b9a2fe4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34219 } }
I20260811 22:09:37.733805 1844 raft_consensus.cc:374] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:37.733881 1844 raft_consensus.cc:729] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b45cf06068124b8bbbdfce7bd20af55a, State: Initialized, Role: FOLLOWER
I20260811 22:09:37.733423 1845 raft_consensus.cc:348] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "862bebb847584850aa16b87c6ebd43f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33787 } } peers { permanent_uuid: "b45cf06068124b8bbbdfce7bd20af55a" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38769 } } peers { permanent_uuid: "e2a6d0152e22481882c084207b9a2fe4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34219 } }
I20260811 22:09:37.734040 1845 raft_consensus.cc:374] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:37.734181 1845 raft_consensus.cc:729] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 862bebb847584850aa16b87c6ebd43f5, State: Initialized, Role: FOLLOWER
I20260811 22:09:37.734764 1844 consensus_queue.cc:260] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "862bebb847584850aa16b87c6ebd43f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33787 } } peers { permanent_uuid: "b45cf06068124b8bbbdfce7bd20af55a" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38769 } } peers { permanent_uuid: "e2a6d0152e22481882c084207b9a2fe4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34219 } }
I20260811 22:09:37.735065 1845 consensus_queue.cc:260] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "862bebb847584850aa16b87c6ebd43f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33787 } } peers { permanent_uuid: "b45cf06068124b8bbbdfce7bd20af55a" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38769 } } peers { permanent_uuid: "e2a6d0152e22481882c084207b9a2fe4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34219 } }
I20260811 22:09:37.736028 1822 heartbeater.cc:503] Master 127.28.47.190:35843 was elected leader, sending a full tablet report...
I20260811 22:09:37.736718 1844 ts_tablet_manager.cc:1428] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a: Time spent starting tablet: real 0.012s user 0.012s sys 0.000s
I20260811 22:09:37.737493 1845 ts_tablet_manager.cc:1428] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5: Time spent starting tablet: real 0.015s user 0.015s sys 0.000s
I20260811 22:09:37.737831 1671 heartbeater.cc:503] Master 127.28.47.190:35843 was elected leader, sending a full tablet report...
W20260811 22:09:37.783259 1515 tablet.cc:2369] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:09:37.824738 1823 tablet.cc:2369] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:09:37.925951 1672 tablet.cc:2369] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:09:37.996475 1851 raft_consensus.cc:482] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:37.996878 1851 raft_consensus.cc:504] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "862bebb847584850aa16b87c6ebd43f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33787 } } peers { permanent_uuid: "b45cf06068124b8bbbdfce7bd20af55a" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38769 } } peers { permanent_uuid: "e2a6d0152e22481882c084207b9a2fe4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34219 } }
I20260811 22:09:37.999109 1851 leader_election.cc:288] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 862bebb847584850aa16b87c6ebd43f5 (127.28.47.130:33787), e2a6d0152e22481882c084207b9a2fe4 (127.28.47.129:34219)
I20260811 22:09:38.005458 1825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.999579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:34219 (local address 127.28.47.131:55243)
0811 22:09:37.999856 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:37.999874 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:37.999994 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:38.000264 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:38.000268 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:38.000288 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:38.000491 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:38.000502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:38.001609 (+ 1107us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:38.001626 (+ 17us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:38.002797 (+ 1171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:38.002816 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:38.004261 (+ 1445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:38.004269 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:38.004587 (+ 318us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:38.004674 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 22:09:38.004890 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 22:09:38.005094 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"mutex_wait_us":5}
I20260811 22:09:38.005944 1841 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.999691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:55243 (local address 127.28.47.129:34219)
0811 22:09:37.999875 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:37.999880 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:37.999988 (+ 108us) server_negotiation.cc:407] Connection header received
0811 22:09:38.000099 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:38.000102 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:38.000154 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:38.000238 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:38.000663 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:38.001444 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:38.003013 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:38.003873 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:38.004139 (+ 266us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:38.004262 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:38.005577 (+ 1315us) server_negotiation.cc:299] Negotiation successful
0811 22:09:38.005752 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:09:38.006660 1838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.999794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:43763 (local address 127.28.47.130:33787)
0811 22:09:37.999939 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:37.999944 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:38.000341 (+ 397us) server_negotiation.cc:407] Connection header received
0811 22:09:38.000617 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:38.000622 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:38.000692 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:38.000820 (+ 128us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:38.001393 (+ 573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:38.002112 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:38.003966 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:38.004988 (+ 1022us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:38.005295 (+ 307us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:38.005425 (+ 130us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:38.006283 (+ 858us) server_negotiation.cc:299] Negotiation successful
0811 22:09:38.006417 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:09:38.006834 1853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:37.999579 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:33787 (local address 127.28.47.131:43763)
0811 22:09:38.000189 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:38.000203 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:38.000420 (+ 217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:38.000848 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:38.000851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:38.000888 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:38.001078 (+ 190us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:38.001086 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:38.002328 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:38.002336 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:38.003635 (+ 1299us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:38.003660 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:38.005671 (+ 2011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:38.005679 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:38.005858 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:38.005897 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:09:38.006271 (+ 374us) client_negotiation.cc:240] Negotiation successful
0811 22:09:38.006434 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":521,"mutex_wait_us":56,"thread_start_us":154,"threads_started":1}
I20260811 22:09:38.007197 1460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a796f0b4e184511a0d0d8e157a6c24f" candidate_uuid: "b45cf06068124b8bbbdfce7bd20af55a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a6d0152e22481882c084207b9a2fe4" is_pre_election: true
I20260811 22:09:38.007985 1460 raft_consensus.cc:2454] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45cf06068124b8bbbdfce7bd20af55a in term 0.
I20260811 22:09:38.008126 1618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a796f0b4e184511a0d0d8e157a6c24f" candidate_uuid: "b45cf06068124b8bbbdfce7bd20af55a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "862bebb847584850aa16b87c6ebd43f5" is_pre_election: true
I20260811 22:09:38.008667 1618 raft_consensus.cc:2454] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b45cf06068124b8bbbdfce7bd20af55a in term 0.
I20260811 22:09:38.009033 1696 leader_election.cc:302] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b45cf06068124b8bbbdfce7bd20af55a, e2a6d0152e22481882c084207b9a2fe4; no voters:
I20260811 22:09:38.009682 1851 raft_consensus.cc:2781] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:38.009881 1851 raft_consensus.cc:482] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:38.010136 1851 raft_consensus.cc:3037] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:38.012377 1851 raft_consensus.cc:504] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "862bebb847584850aa16b87c6ebd43f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33787 } } peers { permanent_uuid: "b45cf06068124b8bbbdfce7bd20af55a" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38769 } } peers { permanent_uuid: "e2a6d0152e22481882c084207b9a2fe4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34219 } }
I20260811 22:09:38.013252 1851 leader_election.cc:288] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [CANDIDATE]: Term 1 election: Requested vote from peers 862bebb847584850aa16b87c6ebd43f5 (127.28.47.130:33787), e2a6d0152e22481882c084207b9a2fe4 (127.28.47.129:34219)
I20260811 22:09:38.014065 1618 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a796f0b4e184511a0d0d8e157a6c24f" candidate_uuid: "b45cf06068124b8bbbdfce7bd20af55a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "862bebb847584850aa16b87c6ebd43f5"
I20260811 22:09:38.014115 1460 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6a796f0b4e184511a0d0d8e157a6c24f" candidate_uuid: "b45cf06068124b8bbbdfce7bd20af55a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2a6d0152e22481882c084207b9a2fe4"
I20260811 22:09:38.014362 1618 raft_consensus.cc:3037] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:38.014395 1460 raft_consensus.cc:3037] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:38.016291 1460 raft_consensus.cc:2454] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45cf06068124b8bbbdfce7bd20af55a in term 1.
I20260811 22:09:38.016321 1618 raft_consensus.cc:2454] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b45cf06068124b8bbbdfce7bd20af55a in term 1.
I20260811 22:09:38.017028 1696 leader_election.cc:302] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 862bebb847584850aa16b87c6ebd43f5, b45cf06068124b8bbbdfce7bd20af55a, e2a6d0152e22481882c084207b9a2fe4; no voters:
I20260811 22:09:38.017383 1851 raft_consensus.cc:2781] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:38.017879 1851 raft_consensus.cc:686] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [term 1 LEADER]: Becoming Leader. State: Replica: b45cf06068124b8bbbdfce7bd20af55a, State: Running, Role: LEADER
I20260811 22:09:38.018599 1851 consensus_queue.cc:237] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "862bebb847584850aa16b87c6ebd43f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33787 } } peers { permanent_uuid: "b45cf06068124b8bbbdfce7bd20af55a" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38769 } } peers { permanent_uuid: "e2a6d0152e22481882c084207b9a2fe4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34219 } }
I20260811 22:09:38.024302 1291 catalog_manager.cc:5495] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a reported cstate change: term changed from 0 to 1, leader changed from <none> to b45cf06068124b8bbbdfce7bd20af55a (127.28.47.131). New cstate: current_term: 1 leader_uuid: "b45cf06068124b8bbbdfce7bd20af55a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "862bebb847584850aa16b87c6ebd43f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33787 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b45cf06068124b8bbbdfce7bd20af55a" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38769 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2a6d0152e22481882c084207b9a2fe4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34219 } health_report { overall_health: UNKNOWN } } }
I20260811 22:09:38.062331 1842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:38.056209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44018 (local address 127.28.47.131:38769)
0811 22:09:38.056392 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:38.056396 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:38.056413 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:09:38.056547 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:38.056550 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:38.056662 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:38.056867 (+ 205us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:38.057422 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:38.058208 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:38.059613 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:38.059931 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:38.060085 (+ 154us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:38.060634 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:38.061618 (+ 984us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:38.061802 (+ 184us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:38.062059 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 22:09:38.062160 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:09:38.073625 1618 raft_consensus.cc:1261] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5 [term 1 FOLLOWER]: Refusing update from remote peer b45cf06068124b8bbbdfce7bd20af55a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:38.073626 1460 raft_consensus.cc:1261] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4 [term 1 FOLLOWER]: Refusing update from remote peer b45cf06068124b8bbbdfce7bd20af55a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:38.074717 1851 consensus_queue.cc:1035] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2a6d0152e22481882c084207b9a2fe4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34219 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:38.075155 1854 consensus_queue.cc:1035] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [LEADER]: Connected to new peer: Peer: permanent_uuid: "862bebb847584850aa16b87c6ebd43f5" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33787 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:38.093389 1860 mvcc.cc:205] Tried to move back new op lower bound from 7317447385376432128 to 7317447385168801792. Current Snapshot: MvccSnapshot[applied={T|T < 7317447385376432128}]
I20260811 22:09:38.111228 1291 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:38.111686 1291 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:38.117774 1291 catalog_manager.cc:5782] T 00000000000000000000000000000000 P caa3dbb32ae843259f3ff8baa0bd1173: Sending DeleteTablet for 3 replicas of tablet 6a796f0b4e184511a0d0d8e157a6c24f
I20260811 22:09:38.119566 1596 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a796f0b4e184511a0d0d8e157a6c24f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:38 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:46154
I20260811 22:09:38.119566 1747 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a796f0b4e184511a0d0d8e157a6c24f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:38 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:44016
I20260811 22:09:38.120672 1871 tablet_replica.cc:330] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5: stopping tablet replica
I20260811 22:09:38.120716 1435 tablet_service.cc:1514] Processing DeleteTablet for tablet 6a796f0b4e184511a0d0d8e157a6c24f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:38 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:43568
I20260811 22:09:38.121482 1871 raft_consensus.cc:2229] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:38.121716 1872 tablet_replica.cc:330] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4: stopping tablet replica
I20260811 22:09:38.122191 1871 raft_consensus.cc:2258] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:38.122493 1872 raft_consensus.cc:2229] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:38.122887 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 1360
I20260811 22:09:38.123015 1870 tablet_replica.cc:330] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a: stopping tablet replica
I20260811 22:09:38.123370 1872 raft_consensus.cc:2258] T 6a796f0b4e184511a0d0d8e157a6c24f P e2a6d0152e22481882c084207b9a2fe4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:38.123574 1870 raft_consensus.cc:2229] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:09:38.124368 1870 raft_consensus.cc:2258] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:38.131978 1870 ts_tablet_manager.cc:1905] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:38.135255 1870 ts_tablet_manager.cc:1918] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:38.135385 1870 log.cc:1196] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/6a796f0b4e184511a0d0d8e157a6c24f
I20260811 22:09:38.135716 1870 ts_tablet_manager.cc:1939] T 6a796f0b4e184511a0d0d8e157a6c24f P b45cf06068124b8bbbdfce7bd20af55a: Deleting consensus metadata
I20260811 22:09:38.137249 1272 catalog_manager.cc:4841] TS b45cf06068124b8bbbdfce7bd20af55a (127.28.47.131:38769): tablet 6a796f0b4e184511a0d0d8e157a6c24f (table test-table [id=5a1ee5e80dce41b3a9a96321941988d9]) successfully deleted
I20260811 22:09:38.138378 1871 ts_tablet_manager.cc:1905] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:38.142652 1871 ts_tablet_manager.cc:1918] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:38.142793 1871 log.cc:1196] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/6a796f0b4e184511a0d0d8e157a6c24f
I20260811 22:09:38.143148 1871 ts_tablet_manager.cc:1939] T 6a796f0b4e184511a0d0d8e157a6c24f P 862bebb847584850aa16b87c6ebd43f5: Deleting consensus metadata
I20260811 22:09:38.145880 1272 catalog_manager.cc:4841] TS 862bebb847584850aa16b87c6ebd43f5 (127.28.47.130:33787): tablet 6a796f0b4e184511a0d0d8e157a6c24f (table test-table [id=5a1ee5e80dce41b3a9a96321941988d9]) successfully deleted
W20260811 22:09:38.151975 1277 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:34219 (error 108)
I20260811 22:09:38.152832 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 1519
I20260811 22:09:38.155936 1837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:38.152989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:34219 (local address 127.0.0.1:43570)
0811 22:09:38.155686 (+ 2697us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:38.155801 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:34219: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2554}
W20260811 22:09:38.165536 1277 catalog_manager.cc:4584] TS e2a6d0152e22481882c084207b9a2fe4 (127.28.47.129:34219): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6a796f0b4e184511a0d0d8e157a6c24f: Network error: Client connection negotiation failed: client connection to 127.28.47.129:34219: connect: Connection refused (error 111)
I20260811 22:09:38.171869 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 1676
I20260811 22:09:38.184273 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 1252
2026-08-11T22:09:38Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (4060 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-11T22:09:38Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:38Z Disabled control of system clock
I20260811 22:09:38.220045 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:34653
--webserver_interface=127.28.47.190
--webserver_port=0
--builtin_ntp_servers=127.28.47.148:45567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:34653
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:09:38.342315 1878 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:38.342653 1878 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:38.342694 1878 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:38.347070 1878 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:38.347153 1878 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:38.347170 1878 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:38.347188 1878 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:38.347204 1878 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:38.352808 1878 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:45567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:34653
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:34653
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1878
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:38.354279 1878 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:38.355969 1878 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:38.365108 1887 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:38.365108 1884 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:38.365137 1890 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:38.365635 1878 server_base.cc:1034] running on GCE node
I20260811 22:09:38.366245 1878 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:38.367478 1878 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:38.368700 1878 hybrid_clock.cc:648] HybridClock initialized: now 1786486178368663 us; error 35 us; skew 500 ppm
I20260811 22:09:38.371284 1878 webserver.cc:460] Webserver started at http://127.28.47.190:39575/ using document root <none> and password file <none>
I20260811 22:09:38.371944 1878 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:38.372025 1878 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:38.372241 1878 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:38.374836 1878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "71f6e7fa5c774d529303c374e02a5efe"
format_stamp: "Formatted at 2026-08-11 22:09:38 on dist-test-slave-vxj2"
server_key: "b2af2c902c1712571d2fbedd697eea36"
server_key_iv: "f3eeb0507424cd4c3e2ae4942e5dccf2"
server_key_version: "encryptionkey@0"
I20260811 22:09:38.375522 1878 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "71f6e7fa5c774d529303c374e02a5efe"
format_stamp: "Formatted at 2026-08-11 22:09:38 on dist-test-slave-vxj2"
server_key: "b2af2c902c1712571d2fbedd697eea36"
server_key_iv: "f3eeb0507424cd4c3e2ae4942e5dccf2"
server_key_version: "encryptionkey@0"
I20260811 22:09:38.380386 1878 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.001s
I20260811 22:09:38.383387 1897 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:38.384912 1878 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:09:38.385041 1878 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "71f6e7fa5c774d529303c374e02a5efe"
format_stamp: "Formatted at 2026-08-11 22:09:38 on dist-test-slave-vxj2"
server_key: "b2af2c902c1712571d2fbedd697eea36"
server_key_iv: "f3eeb0507424cd4c3e2ae4942e5dccf2"
server_key_version: "encryptionkey@0"
I20260811 22:09:38.385289 1878 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:38.390683 1878 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:38.391363 1878 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:38.391525 1878 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:38.398985 1878 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:34653
I20260811 22:09:38.398988 1958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:34653 every 8 connection(s)
I20260811 22:09:38.400108 1878 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:38.403805 1959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:38.406889 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 1878
I20260811 22:09:38.407023 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:38.407327 28862 external_mini_cluster.cc:1428] Setting key 988506ba063d387d370594f74354c01c
I20260811 22:09:38.411830 1959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe: Bootstrap starting.
I20260811 22:09:38.413945 1962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:38.408851 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40712 (local address 127.28.47.190:34653)
0811 22:09:38.409697 (+ 846us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:38.409721 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:38.409764 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:09:38.410906 (+ 1142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:38.410932 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:38.411363 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:38.411749 (+ 386us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:38.412552 (+ 803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:38.412557 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:38.412569 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:38.412578 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:38.412628 (+ 50us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:38.412644 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:38.412807 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:38.412886 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 22:09:38.413145 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":369,"thread_start_us":121,"threads_started":1}
I20260811 22:09:38.414786 1959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:38.415606 1959 log.cc:824] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:38.417752 1959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe: No bootstrap required, opened a new log
I20260811 22:09:38.422544 1959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f6e7fa5c774d529303c374e02a5efe" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34653 } }
I20260811 22:09:38.422947 1959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:38.423013 1959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 71f6e7fa5c774d529303c374e02a5efe, State: Initialized, Role: FOLLOWER
I20260811 22:09:38.423604 1959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f6e7fa5c774d529303c374e02a5efe" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34653 } }
I20260811 22:09:38.423799 1959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:38.423887 1959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:38.424016 1959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:38.425849 1959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f6e7fa5c774d529303c374e02a5efe" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34653 } }
I20260811 22:09:38.426494 1959 leader_election.cc:302] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 71f6e7fa5c774d529303c374e02a5efe; no voters:
I20260811 22:09:38.427043 1959 leader_election.cc:288] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:38.427175 1964 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:38.427448 1964 raft_consensus.cc:686] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [term 1 LEADER]: Becoming Leader. State: Replica: 71f6e7fa5c774d529303c374e02a5efe, State: Running, Role: LEADER
I20260811 22:09:38.427820 1964 consensus_queue.cc:237] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f6e7fa5c774d529303c374e02a5efe" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34653 } }
I20260811 22:09:38.428736 1959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:38.430758 1967 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 71f6e7fa5c774d529303c374e02a5efe. Latest consensus state: current_term: 1 leader_uuid: "71f6e7fa5c774d529303c374e02a5efe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f6e7fa5c774d529303c374e02a5efe" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34653 } } }
I20260811 22:09:38.430914 1967 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:38.431141 1966 sys_catalog.cc:455] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "71f6e7fa5c774d529303c374e02a5efe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "71f6e7fa5c774d529303c374e02a5efe" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34653 } } }
I20260811 22:09:38.431254 1966 sys_catalog.cc:458] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:38.431577 1972 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:38.435447 1972 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:38.442308 1972 catalog_manager.cc:1347] Generated new cluster ID: 853d512a3bd04465bbb0c363d7b1b1fa
I20260811 22:09:38.442387 1972 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:38.454203 1972 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:38.454636 1972 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:38.460891 1972 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe: Generated new TSK 0
I20260811 22:09:38.461791 1972 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:09:38.471129 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:34653
--builtin_ntp_servers=127.28.47.148:45567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260811 22:09:38.594285 1985 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:38.594595 1985 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:38.594640 1985 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:38.599155 1985 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:38.599262 1985 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:38.599364 1985 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:38.604918 1985 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:45567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:34653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1985
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:38.606281 1985 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:38.607913 1985 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:38.618790 1992 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:38.618883 1991 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:38.618894 1994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:38.619334 1985 server_base.cc:1034] running on GCE node
I20260811 22:09:38.619827 1985 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:38.620605 1985 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:38.621845 1985 hybrid_clock.cc:648] HybridClock initialized: now 1786486178621814 us; error 39 us; skew 500 ppm
I20260811 22:09:38.624161 1985 webserver.cc:460] Webserver started at http://127.28.47.129:43701/ using document root <none> and password file <none>
I20260811 22:09:38.624953 1985 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:38.625052 1985 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:38.625310 1985 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:38.627573 1985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "4f79e9ce91aa4402af85747fc3451c26"
format_stamp: "Formatted at 2026-08-11 22:09:38 on dist-test-slave-vxj2"
server_key: "8c91da3299f3d4f1032731dde105f6ac"
server_key_iv: "f9bf268f97999f1647ee5bc23fc48f4a"
server_key_version: "encryptionkey@0"
I20260811 22:09:38.628216 1985 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "4f79e9ce91aa4402af85747fc3451c26"
format_stamp: "Formatted at 2026-08-11 22:09:38 on dist-test-slave-vxj2"
server_key: "8c91da3299f3d4f1032731dde105f6ac"
server_key_iv: "f9bf268f97999f1647ee5bc23fc48f4a"
server_key_version: "encryptionkey@0"
I20260811 22:09:38.632572 1985 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:09:38.635597 2001 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:38.636986 1985 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:38.637197 1985 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "4f79e9ce91aa4402af85747fc3451c26"
format_stamp: "Formatted at 2026-08-11 22:09:38 on dist-test-slave-vxj2"
server_key: "8c91da3299f3d4f1032731dde105f6ac"
server_key_iv: "f9bf268f97999f1647ee5bc23fc48f4a"
server_key_version: "encryptionkey@0"
I20260811 22:09:38.637355 1985 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:38.642432 1985 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:38.643139 1985 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:38.643343 1985 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:38.643946 1985 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:38.645125 1985 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:38.645246 1985 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:38.645323 1985 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:38.645371 1985 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:38.655167 1985 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:33523
I20260811 22:09:38.655196 2141 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:33523 every 8 connection(s)
I20260811 22:09:38.656217 1985 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:38.659245 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 1985
I20260811 22:09:38.659392 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:09:38.659715 28862 external_mini_cluster.cc:1428] Setting key a6bbf018b3d9fedb290d1bf7cb2fdc86
I20260811 22:09:38.662361 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:34653
--builtin_ntp_servers=127.28.47.148:45567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:09:38.663825 1962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:38.658347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:55487 (local address 127.28.47.190:34653)
0811 22:09:38.658538 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:38.658542 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:38.659886 (+ 1344us) server_negotiation.cc:407] Connection header received
0811 22:09:38.661670 (+ 1784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:38.661673 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:38.661709 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:38.661777 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:38.663258 (+ 1481us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:38.663265 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:38.663271 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:38.663290 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:38.663319 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:38.663332 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:38.663432 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:38.663581 (+ 149us) server_negotiation.cc:299] Negotiation successful
0811 22:09:38.663634 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:09:38.665282 2144 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:38.658752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:34653 (local address 127.28.47.129:55487)
0811 22:09:38.659649 (+ 897us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:38.659711 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:38.661435 (+ 1724us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:38.662015 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:38.662024 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:38.662149 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:38.662976 (+ 827us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:09:38.662977 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:38.663021 (+ 44us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:09:38.663093 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:38.663433 (+ 340us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:38.663442 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:38.663455 (+ 13us) client_negotiation.cc:769] Sending connection context
0811 22:09:38.663593 (+ 138us) client_negotiation.cc:240] Negotiation successful
0811 22:09:38.663856 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":324,"thread_start_us":134,"threads_started":1}
I20260811 22:09:38.666704 2142 heartbeater.cc:347] Connected to a master server at 127.28.47.190:34653
I20260811 22:09:38.666961 2142 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:38.667426 2142 heartbeater.cc:511] Master 127.28.47.190:34653 requested a full tablet report, sending...
I20260811 22:09:38.669377 1917 ts_manager.cc:194] Registered new tserver with Master: 4f79e9ce91aa4402af85747fc3451c26 (127.28.47.129:33523)
W20260811 22:09:38.782450 2145 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:38.782763 2145 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:38.782806 2145 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:38.786985 2145 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:38.787055 2145 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:38.787114 2145 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:38.792081 2145 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:45567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:34653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2145
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:38.793231 2145 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:38.794675 2145 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:38.804911 2152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:38.805034 2145 server_base.cc:1034] running on GCE node
W20260811 22:09:38.804912 2156 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:38.804991 2153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:38.805663 2145 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:38.806366 2145 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:38.807564 2145 hybrid_clock.cc:648] HybridClock initialized: now 1786486178807544 us; error 31 us; skew 500 ppm
I20260811 22:09:38.809485 2145 webserver.cc:460] Webserver started at http://127.28.47.130:40379/ using document root <none> and password file <none>
I20260811 22:09:38.810165 2145 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:38.810256 2145 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:38.810505 2145 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:38.812702 2145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "7e44914064eb470b83699be7e2c9659c"
format_stamp: "Formatted at 2026-08-11 22:09:38 on dist-test-slave-vxj2"
server_key: "95aae9b93c4d1b68cdcc1f8563045524"
server_key_iv: "cab2f9bccd2a4269750427d40974359b"
server_key_version: "encryptionkey@0"
I20260811 22:09:38.813386 2145 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "7e44914064eb470b83699be7e2c9659c"
format_stamp: "Formatted at 2026-08-11 22:09:38 on dist-test-slave-vxj2"
server_key: "95aae9b93c4d1b68cdcc1f8563045524"
server_key_iv: "cab2f9bccd2a4269750427d40974359b"
server_key_version: "encryptionkey@0"
I20260811 22:09:38.817476 2145 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:09:38.820251 2165 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:38.821681 2145 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:38.821822 2145 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "7e44914064eb470b83699be7e2c9659c"
format_stamp: "Formatted at 2026-08-11 22:09:38 on dist-test-slave-vxj2"
server_key: "95aae9b93c4d1b68cdcc1f8563045524"
server_key_iv: "cab2f9bccd2a4269750427d40974359b"
server_key_version: "encryptionkey@0"
I20260811 22:09:38.821957 2145 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:38.826862 2145 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:38.827530 2145 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:38.827723 2145 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:38.828333 2145 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:38.829470 2145 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:38.829536 2145 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:38.829607 2145 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:38.829653 2145 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:38.839370 2145 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:35815
I20260811 22:09:38.839408 2296 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:35815 every 8 connection(s)
I20260811 22:09:38.840452 2145 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:38.840638 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 2145
I20260811 22:09:38.840751 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:09:38.841107 28862 external_mini_cluster.cc:1428] Setting key bf80c39316673142e7e635af492e7f0e
I20260811 22:09:38.843391 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:34653
--builtin_ntp_servers=127.28.47.148:45567
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260811 22:09:38.848080 1962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:38.842882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:52699 (local address 127.28.47.190:34653)
0811 22:09:38.843059 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:38.843064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:38.844313 (+ 1249us) server_negotiation.cc:407] Connection header received
0811 22:09:38.846174 (+ 1861us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:38.846176 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:38.846218 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:38.846282 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:38.847587 (+ 1305us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:38.847590 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:38.847592 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:38.847604 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:38.847625 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:38.847633 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:38.847721 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:38.847907 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 22:09:38.847967 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:09:38.849548 2299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:38.843275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:34653 (local address 127.28.47.130:52699)
0811 22:09:38.844135 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:38.844193 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:38.845914 (+ 1721us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:38.846494 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:38.846506 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:38.846626 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:38.847249 (+ 623us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:09:38.847254 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:38.847313 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:09:38.847417 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:38.847721 (+ 304us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:38.847733 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:38.847742 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:09:38.847969 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 22:09:38.848251 (+ 282us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":161,"threads_started":1}
I20260811 22:09:38.850852 2297 heartbeater.cc:347] Connected to a master server at 127.28.47.190:34653
I20260811 22:09:38.851157 2297 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:38.851770 2297 heartbeater.cc:511] Master 127.28.47.190:34653 requested a full tablet report, sending...
I20260811 22:09:38.853108 1917 ts_manager.cc:194] Registered new tserver with Master: 7e44914064eb470b83699be7e2c9659c (127.28.47.130:35815)
W20260811 22:09:38.966514 2300 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:38.966843 2300 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:38.966945 2300 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:38.971125 2300 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:38.971220 2300 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:38.971310 2300 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:38.976475 2300 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:45567
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:34653
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2300
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:38.977843 2300 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:38.979395 2300 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:38.990316 2308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:38.990530 2300 server_base.cc:1034] running on GCE node
W20260811 22:09:38.990309 2311 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:38.990316 2309 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:38.991227 2300 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:38.991932 2300 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:38.993237 2300 hybrid_clock.cc:648] HybridClock initialized: now 1786486178993138 us; error 40 us; skew 500 ppm
I20260811 22:09:38.995368 2300 webserver.cc:460] Webserver started at http://127.28.47.131:33433/ using document root <none> and password file <none>
I20260811 22:09:38.996090 2300 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:38.996161 2300 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:38.996387 2300 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:38.998713 2300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "58bf0a4315cd422498fb315258f8adc6"
format_stamp: "Formatted at 2026-08-11 22:09:38 on dist-test-slave-vxj2"
server_key: "73a6604c97961280150cbf44db484526"
server_key_iv: "f4a1453f353211b4e934661047e24eb6"
server_key_version: "encryptionkey@0"
I20260811 22:09:38.999316 2300 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "58bf0a4315cd422498fb315258f8adc6"
format_stamp: "Formatted at 2026-08-11 22:09:38 on dist-test-slave-vxj2"
server_key: "73a6604c97961280150cbf44db484526"
server_key_iv: "f4a1453f353211b4e934661047e24eb6"
server_key_version: "encryptionkey@0"
I20260811 22:09:39.003510 2300 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:09:39.006465 2317 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:39.007884 2300 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:09:39.008035 2300 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "58bf0a4315cd422498fb315258f8adc6"
format_stamp: "Formatted at 2026-08-11 22:09:38 on dist-test-slave-vxj2"
server_key: "73a6604c97961280150cbf44db484526"
server_key_iv: "f4a1453f353211b4e934661047e24eb6"
server_key_version: "encryptionkey@0"
I20260811 22:09:39.008167 2300 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:39.013266 2300 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:39.013931 2300 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:39.014154 2300 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:39.014750 2300 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:39.015735 2300 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:39.015813 2300 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:39.015887 2300 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:39.015937 2300 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:39.025520 2300 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:43691
I20260811 22:09:39.025556 2448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:43691 every 8 connection(s)
I20260811 22:09:39.026645 2300 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:39.031927 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 2300
I20260811 22:09:39.032090 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:09:39.032384 28862 external_mini_cluster.cc:1428] Setting key 598c4a66bdbc38aa3f26956ef1626f0c
I20260811 22:09:39.033957 1962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.028690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:37337 (local address 127.28.47.190:34653)
0811 22:09:39.028867 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:39.028872 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:39.030252 (+ 1380us) server_negotiation.cc:407] Connection header received
0811 22:09:39.031765 (+ 1513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:39.031768 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:39.031821 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:39.031928 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:39.033454 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:39.033458 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:39.033466 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:39.033479 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:39.033503 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:39.033511 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:39.033611 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:39.033764 (+ 153us) server_negotiation.cc:299] Negotiation successful
0811 22:09:39.033816 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:09:39.034950 2452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.029261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:34653 (local address 127.28.47.131:37337)
0811 22:09:39.030091 (+ 830us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:39.030137 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:39.031470 (+ 1333us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:39.032168 (+ 698us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:39.032189 (+ 21us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:39.032310 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:39.033053 (+ 743us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:09:39.033070 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:39.033136 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:09:39.033267 (+ 131us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:39.033598 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:39.033608 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:39.033615 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:09:39.033819 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 22:09:39.034116 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":128,"threads_started":1}
I20260811 22:09:39.036068 2449 heartbeater.cc:347] Connected to a master server at 127.28.47.190:34653
I20260811 22:09:39.036298 2449 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:39.036715 2449 heartbeater.cc:511] Master 127.28.47.190:34653 requested a full tablet report, sending...
I20260811 22:09:39.037750 1917 ts_manager.cc:194] Registered new tserver with Master: 58bf0a4315cd422498fb315258f8adc6 (127.28.47.131:43691)
I20260811 22:09:39.046604 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:09:39.050309 1962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.048583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40726 (local address 127.28.47.190:34653)
0811 22:09:39.048754 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:39.048759 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:39.048946 (+ 187us) server_negotiation.cc:407] Connection header received
0811 22:09:39.049035 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:39.049039 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:39.049100 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:39.049214 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:39.049910 (+ 696us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:39.049913 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:39.049915 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:39.049926 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:39.049948 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:39.049956 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:39.050051 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:39.050078 (+ 27us) server_negotiation.cc:299] Negotiation successful
0811 22:09:39.050113 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:09:39.053483 1917 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:40726:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:09:39.056710 1917 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:39.069741 2470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.066814 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37534 (local address 127.28.47.131:43691)
0811 22:09:39.067373 (+ 559us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:39.067386 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:39.067411 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:09:39.067465 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:39.067471 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:39.067669 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:39.068253 (+ 584us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:39.069144 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:39.069148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:39.069182 (+ 34us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:39.069192 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:39.069226 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:39.069254 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:39.069417 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:39.069528 (+ 111us) server_negotiation.cc:299] Negotiation successful
0811 22:09:39.069602 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":467,"thread_start_us":167,"threads_started":1}
I20260811 22:09:39.069725 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.065978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:35815 (local address 127.0.0.1:45684)
0811 22:09:39.066942 (+ 964us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:39.067008 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:39.067114 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:39.067857 (+ 743us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:39.067859 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:39.067936 (+ 77us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:39.068682 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:09:39.068684 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:39.068692 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:09:39.068750 (+ 58us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:39.069372 (+ 622us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:39.069383 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:39.069389 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:09:39.069483 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:09:39.069560 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":726,"thread_start_us":81,"threads_started":1}
I20260811 22:09:39.069746 2468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.066176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:43691 (local address 127.0.0.1:37534)
0811 22:09:39.066941 (+ 765us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:39.067001 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:39.067116 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:39.067848 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:39.067855 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:39.067936 (+ 81us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:39.068654 (+ 718us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:09:39.068657 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:39.068679 (+ 22us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:09:39.068750 (+ 71us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:39.069382 (+ 632us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:39.069385 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:39.069394 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:09:39.069513 (+ 119us) client_negotiation.cc:240] Negotiation successful
0811 22:09:39.069571 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":3,"thread_start_us":151,"threads_started":1}
I20260811 22:09:39.069986 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.066823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49900 (local address 127.28.47.129:33523)
0811 22:09:39.067388 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:39.067400 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:39.067600 (+ 200us) server_negotiation.cc:407] Connection header received
0811 22:09:39.067794 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:39.067801 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:39.067971 (+ 170us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:39.068066 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:39.069372 (+ 1306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:39.069378 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:39.069389 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:39.069402 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:39.069446 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:39.069485 (+ 39us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:39.069584 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:39.069754 (+ 170us) server_negotiation.cc:299] Negotiation successful
0811 22:09:39.069829 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462,"thread_start_us":175,"threads_started":1}
I20260811 22:09:39.070362 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.066408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:33523 (local address 127.0.0.1:49900)
0811 22:09:39.067511 (+ 1103us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:39.067524 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:39.067654 (+ 130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:39.068764 (+ 1110us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:39.068767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:39.068779 (+ 12us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:39.069202 (+ 423us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:09:39.069204 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:39.069215 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:09:39.069250 (+ 35us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:39.069625 (+ 375us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:39.069628 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:39.069632 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:09:39.070198 (+ 566us) client_negotiation.cc:240] Negotiation successful
0811 22:09:39.070237 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1048,"thread_start_us":129,"threads_started":1}
I20260811 22:09:39.070428 2467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.066160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45684 (local address 127.28.47.130:35815)
0811 22:09:39.067373 (+ 1213us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:39.067386 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:39.067411 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:09:39.067466 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:39.067471 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:39.067669 (+ 198us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:39.068253 (+ 584us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:39.069129 (+ 876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:39.069135 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:39.069173 (+ 38us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:39.069184 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:39.069227 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:39.069253 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:39.070147 (+ 894us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:39.070195 (+ 48us) server_negotiation.cc:299] Negotiation successful
0811 22:09:39.070270 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1099,"thread_start_us":148,"threads_started":1}
I20260811 22:09:39.073107 2374 tablet_service.cc:1467] Processing CreateTablet for tablet 37903598fb634169bfce4b560e6921e2 (DEFAULT_TABLE table=test-table [id=bbc8991bd70344f08ed9430936901f48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:39.073256 2220 tablet_service.cc:1467] Processing CreateTablet for tablet 37903598fb634169bfce4b560e6921e2 (DEFAULT_TABLE table=test-table [id=bbc8991bd70344f08ed9430936901f48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:39.073263 2064 tablet_service.cc:1467] Processing CreateTablet for tablet 37903598fb634169bfce4b560e6921e2 (DEFAULT_TABLE table=test-table [id=bbc8991bd70344f08ed9430936901f48]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:39.074373 2064 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37903598fb634169bfce4b560e6921e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:39.074373 2220 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37903598fb634169bfce4b560e6921e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:39.074393 2374 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 37903598fb634169bfce4b560e6921e2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:39.080687 2472 tablet_bootstrap.cc:492] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6: Bootstrap starting.
I20260811 22:09:39.082248 2473 tablet_bootstrap.cc:492] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c: Bootstrap starting.
I20260811 22:09:39.083405 2472 tablet_bootstrap.cc:654] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:39.084326 2472 log.cc:824] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:39.084491 2473 tablet_bootstrap.cc:654] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:39.085198 2474 tablet_bootstrap.cc:492] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26: Bootstrap starting.
I20260811 22:09:39.085384 2473 log.cc:824] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:39.086638 2472 tablet_bootstrap.cc:492] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6: No bootstrap required, opened a new log
I20260811 22:09:39.086865 2472 ts_tablet_manager.cc:1397] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:09:39.087394 2474 tablet_bootstrap.cc:654] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:39.087469 2473 tablet_bootstrap.cc:492] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c: No bootstrap required, opened a new log
I20260811 22:09:39.087669 2473 ts_tablet_manager.cc:1397] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 22:09:39.088255 2474 log.cc:824] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:39.090510 2474 tablet_bootstrap.cc:492] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26: No bootstrap required, opened a new log
I20260811 22:09:39.090705 2474 ts_tablet_manager.cc:1397] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.002s
I20260811 22:09:39.091738 2473 raft_consensus.cc:348] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e44914064eb470b83699be7e2c9659c" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 35815 } } peers { permanent_uuid: "58bf0a4315cd422498fb315258f8adc6" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 43691 } } peers { permanent_uuid: "4f79e9ce91aa4402af85747fc3451c26" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33523 } }
I20260811 22:09:39.091715 2472 raft_consensus.cc:348] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e44914064eb470b83699be7e2c9659c" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 35815 } } peers { permanent_uuid: "58bf0a4315cd422498fb315258f8adc6" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 43691 } } peers { permanent_uuid: "4f79e9ce91aa4402af85747fc3451c26" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33523 } }
I20260811 22:09:39.091998 2472 raft_consensus.cc:374] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:39.092003 2473 raft_consensus.cc:374] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:39.092056 2473 raft_consensus.cc:729] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e44914064eb470b83699be7e2c9659c, State: Initialized, Role: FOLLOWER
I20260811 22:09:39.092056 2472 raft_consensus.cc:729] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 58bf0a4315cd422498fb315258f8adc6, State: Initialized, Role: FOLLOWER
I20260811 22:09:39.092531 2473 consensus_queue.cc:260] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e44914064eb470b83699be7e2c9659c" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 35815 } } peers { permanent_uuid: "58bf0a4315cd422498fb315258f8adc6" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 43691 } } peers { permanent_uuid: "4f79e9ce91aa4402af85747fc3451c26" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33523 } }
I20260811 22:09:39.092535 2472 consensus_queue.cc:260] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e44914064eb470b83699be7e2c9659c" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 35815 } } peers { permanent_uuid: "58bf0a4315cd422498fb315258f8adc6" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 43691 } } peers { permanent_uuid: "4f79e9ce91aa4402af85747fc3451c26" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33523 } }
I20260811 22:09:39.093261 2297 heartbeater.cc:503] Master 127.28.47.190:34653 was elected leader, sending a full tablet report...
I20260811 22:09:39.093640 2473 ts_tablet_manager.cc:1428] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 22:09:39.093664 2472 ts_tablet_manager.cc:1428] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:09:39.094043 2449 heartbeater.cc:503] Master 127.28.47.190:34653 was elected leader, sending a full tablet report...
I20260811 22:09:39.095309 2474 raft_consensus.cc:348] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e44914064eb470b83699be7e2c9659c" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 35815 } } peers { permanent_uuid: "58bf0a4315cd422498fb315258f8adc6" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 43691 } } peers { permanent_uuid: "4f79e9ce91aa4402af85747fc3451c26" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33523 } }
I20260811 22:09:39.095549 2474 raft_consensus.cc:374] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:39.095584 2474 raft_consensus.cc:729] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f79e9ce91aa4402af85747fc3451c26, State: Initialized, Role: FOLLOWER
I20260811 22:09:39.096027 2474 consensus_queue.cc:260] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e44914064eb470b83699be7e2c9659c" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 35815 } } peers { permanent_uuid: "58bf0a4315cd422498fb315258f8adc6" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 43691 } } peers { permanent_uuid: "4f79e9ce91aa4402af85747fc3451c26" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33523 } }
I20260811 22:09:39.096964 2142 heartbeater.cc:503] Master 127.28.47.190:34653 was elected leader, sending a full tablet report...
I20260811 22:09:39.097290 2474 ts_tablet_manager.cc:1428] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26: Time spent starting tablet: real 0.007s user 0.004s sys 0.002s
W20260811 22:09:39.157534 2143 tablet.cc:2369] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:09:39.277798 2450 tablet.cc:2369] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:09:39.304877 2480 raft_consensus.cc:482] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:39.305213 2480 raft_consensus.cc:504] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e44914064eb470b83699be7e2c9659c" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 35815 } } peers { permanent_uuid: "58bf0a4315cd422498fb315258f8adc6" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 43691 } } peers { permanent_uuid: "4f79e9ce91aa4402af85747fc3451c26" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33523 } }
I20260811 22:09:39.306510 2480 leader_election.cc:288] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7e44914064eb470b83699be7e2c9659c (127.28.47.130:35815), 58bf0a4315cd422498fb315258f8adc6 (127.28.47.131:43691)
I20260811 22:09:39.314497 2467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.306857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:46299 (local address 127.28.47.130:35815)
0811 22:09:39.306996 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:39.307001 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:39.307199 (+ 198us) server_negotiation.cc:407] Connection header received
0811 22:09:39.307344 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:39.307347 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:39.307401 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:39.307489 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:39.314051 (+ 6562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:39.314054 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:39.314056 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:39.314072 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:39.314096 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:39.314105 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:39.314231 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:39.314297 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 22:09:39.314346 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:09:39.314517 2481 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.306724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:35815 (local address 127.28.47.129:46299)
0811 22:09:39.307085 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:39.307098 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:39.307233 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:39.307513 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:39.307516 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:39.307531 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:39.313769 (+ 6238us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:09:39.313772 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:39.313789 (+ 17us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:09:39.313837 (+ 48us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:39.314206 (+ 369us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:39.314210 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:39.314214 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:09:39.314295 (+ 81us) client_negotiation.cc:240] Negotiation successful
0811 22:09:39.314346 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":115,"threads_started":1}
I20260811 22:09:39.314632 2470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.306930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:56979 (local address 127.28.47.131:43691)
0811 22:09:39.307060 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:39.307065 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:39.307223 (+ 158us) server_negotiation.cc:407] Connection header received
0811 22:09:39.307379 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:39.307382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:39.307417 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:39.307506 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:39.314060 (+ 6554us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:39.314064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:39.314067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:39.314084 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:39.314110 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:39.314120 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:39.314281 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:39.314396 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 22:09:39.314447 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:09:39.314723 2483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.306802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:43691 (local address 127.28.47.129:56979)
0811 22:09:39.307131 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:39.307144 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:39.307252 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:39.307527 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:39.307530 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:39.307546 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:39.313815 (+ 6269us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:09:39.313818 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:39.313832 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:09:39.313870 (+ 38us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:39.314268 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:39.314272 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:39.314276 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:09:39.314397 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 22:09:39.314446 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":72,"threads_started":1}
I20260811 22:09:39.315465 2398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37903598fb634169bfce4b560e6921e2" candidate_uuid: "4f79e9ce91aa4402af85747fc3451c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58bf0a4315cd422498fb315258f8adc6" is_pre_election: true
I20260811 22:09:39.315465 2243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37903598fb634169bfce4b560e6921e2" candidate_uuid: "4f79e9ce91aa4402af85747fc3451c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e44914064eb470b83699be7e2c9659c" is_pre_election: true
I20260811 22:09:39.315876 2398 raft_consensus.cc:2454] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f79e9ce91aa4402af85747fc3451c26 in term 0.
I20260811 22:09:39.315877 2243 raft_consensus.cc:2454] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4f79e9ce91aa4402af85747fc3451c26 in term 0.
I20260811 22:09:39.316507 2004 leader_election.cc:302] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4f79e9ce91aa4402af85747fc3451c26, 58bf0a4315cd422498fb315258f8adc6, 7e44914064eb470b83699be7e2c9659c; no voters:
I20260811 22:09:39.316834 2480 raft_consensus.cc:2781] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:39.316932 2480 raft_consensus.cc:482] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:39.316977 2480 raft_consensus.cc:3037] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:39.318118 2480 raft_consensus.cc:504] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e44914064eb470b83699be7e2c9659c" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 35815 } } peers { permanent_uuid: "58bf0a4315cd422498fb315258f8adc6" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 43691 } } peers { permanent_uuid: "4f79e9ce91aa4402af85747fc3451c26" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33523 } }
I20260811 22:09:39.318532 2480 leader_election.cc:288] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [CANDIDATE]: Term 1 election: Requested vote from peers 7e44914064eb470b83699be7e2c9659c (127.28.47.130:35815), 58bf0a4315cd422498fb315258f8adc6 (127.28.47.131:43691)
I20260811 22:09:39.318941 2243 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37903598fb634169bfce4b560e6921e2" candidate_uuid: "4f79e9ce91aa4402af85747fc3451c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7e44914064eb470b83699be7e2c9659c"
I20260811 22:09:39.319031 2398 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "37903598fb634169bfce4b560e6921e2" candidate_uuid: "4f79e9ce91aa4402af85747fc3451c26" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "58bf0a4315cd422498fb315258f8adc6"
I20260811 22:09:39.319099 2243 raft_consensus.cc:3037] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:39.319162 2398 raft_consensus.cc:3037] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:39.320577 2398 raft_consensus.cc:2454] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f79e9ce91aa4402af85747fc3451c26 in term 1.
I20260811 22:09:39.320577 2243 raft_consensus.cc:2454] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4f79e9ce91aa4402af85747fc3451c26 in term 1.
I20260811 22:09:39.320950 2004 leader_election.cc:302] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 4f79e9ce91aa4402af85747fc3451c26, 58bf0a4315cd422498fb315258f8adc6, 7e44914064eb470b83699be7e2c9659c; no voters:
I20260811 22:09:39.321194 2480 raft_consensus.cc:2781] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:39.321476 2480 raft_consensus.cc:686] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [term 1 LEADER]: Becoming Leader. State: Replica: 4f79e9ce91aa4402af85747fc3451c26, State: Running, Role: LEADER
I20260811 22:09:39.321882 2480 consensus_queue.cc:237] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e44914064eb470b83699be7e2c9659c" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 35815 } } peers { permanent_uuid: "58bf0a4315cd422498fb315258f8adc6" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 43691 } } peers { permanent_uuid: "4f79e9ce91aa4402af85747fc3451c26" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33523 } }
I20260811 22:09:39.325434 1915 catalog_manager.cc:5495] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 reported cstate change: term changed from 0 to 1, leader changed from <none> to 4f79e9ce91aa4402af85747fc3451c26 (127.28.47.129). New cstate: current_term: 1 leader_uuid: "4f79e9ce91aa4402af85747fc3451c26" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7e44914064eb470b83699be7e2c9659c" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 35815 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "58bf0a4315cd422498fb315258f8adc6" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 43691 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4f79e9ce91aa4402af85747fc3451c26" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33523 } health_report { overall_health: HEALTHY } } }
W20260811 22:09:39.342069 2298 tablet.cc:2369] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:09:39.365649 2471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.363922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49908 (local address 127.28.47.129:33523)
0811 22:09:39.364041 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:39.364044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:39.364059 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:09:39.364146 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:39.364148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:39.364210 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:39.364282 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:39.365220 (+ 938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:39.365223 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:39.365225 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:39.365254 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:39.365274 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:39.365282 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:39.365365 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:39.365496 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 22:09:39.365537 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 22:09:39.372083 2398 raft_consensus.cc:1261] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6 [term 1 FOLLOWER]: Refusing update from remote peer 4f79e9ce91aa4402af85747fc3451c26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:39.372084 2243 raft_consensus.cc:1261] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c [term 1 FOLLOWER]: Refusing update from remote peer 4f79e9ce91aa4402af85747fc3451c26: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:39.372897 2484 consensus_queue.cc:1035] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "58bf0a4315cd422498fb315258f8adc6" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 43691 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:39.373143 2480 consensus_queue.cc:1035] T 37903598fb634169bfce4b560e6921e2 P 4f79e9ce91aa4402af85747fc3451c26 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7e44914064eb470b83699be7e2c9659c" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 35815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:39.381253 2489 mvcc.cc:205] Tried to move back new op lower bound from 7317447390700232704 to 7317447390505439232. Current Snapshot: MvccSnapshot[applied={T|T < 7317447390700232704}]
I20260811 22:09:39.399365 1917 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:40726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:39.399559 1917 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:40726:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:39.402743 1917 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 71f6e7fa5c774d529303c374e02a5efe: Sending DeleteTablet for 3 replicas of tablet 37903598fb634169bfce4b560e6921e2
I20260811 22:09:39.403707 2374 tablet_service.cc:1514] Processing DeleteTablet for tablet 37903598fb634169bfce4b560e6921e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:39 UTC) from {username='slave'} at 127.0.0.1:37534
I20260811 22:09:39.403753 2220 tablet_service.cc:1514] Processing DeleteTablet for tablet 37903598fb634169bfce4b560e6921e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:39 UTC) from {username='slave'} at 127.0.0.1:45684
I20260811 22:09:39.404292 2502 tablet_replica.cc:330] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6: stopping tablet replica
I20260811 22:09:39.404524 2064 tablet_service.cc:1514] Processing DeleteTablet for tablet 37903598fb634169bfce4b560e6921e2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:39 UTC) from {username='slave'} at 127.0.0.1:49900
I20260811 22:09:39.404800 2502 raft_consensus.cc:2229] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:39.405275 2502 raft_consensus.cc:2258] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:39.405755 2504 tablet_replica.cc:330] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c: stopping tablet replica
I20260811 22:09:39.405993 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 1985
I20260811 22:09:39.407147 2502 ts_tablet_manager.cc:1905] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:39.409795 2502 ts_tablet_manager.cc:1918] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:39.409981 2502 log.cc:1196] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/37903598fb634169bfce4b560e6921e2
I20260811 22:09:39.410084 2504 raft_consensus.cc:2229] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:39.410408 2502 ts_tablet_manager.cc:1939] T 37903598fb634169bfce4b560e6921e2 P 58bf0a4315cd422498fb315258f8adc6: Deleting consensus metadata
I20260811 22:09:39.410652 2504 raft_consensus.cc:2258] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:39.413376 2504 ts_tablet_manager.cc:1905] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:39.414078 1902 catalog_manager.cc:4841] TS 58bf0a4315cd422498fb315258f8adc6 (127.28.47.131:43691): tablet 37903598fb634169bfce4b560e6921e2 (table test-table [id=bbc8991bd70344f08ed9430936901f48]) successfully deleted
I20260811 22:09:39.417542 2504 ts_tablet_manager.cc:1918] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:39.417701 2504 log.cc:1196] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/37903598fb634169bfce4b560e6921e2
I20260811 22:09:39.418186 2504 ts_tablet_manager.cc:1939] T 37903598fb634169bfce4b560e6921e2 P 7e44914064eb470b83699be7e2c9659c: Deleting consensus metadata
I20260811 22:09:39.419859 1901 catalog_manager.cc:4841] TS 7e44914064eb470b83699be7e2c9659c (127.28.47.130:35815): tablet 37903598fb634169bfce4b560e6921e2 (table test-table [id=bbc8991bd70344f08ed9430936901f48]) successfully deleted
W20260811 22:09:39.426733 1899 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:33523 (error 108)
I20260811 22:09:39.427217 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 2145
I20260811 22:09:39.430672 2465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.430280 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:33523 (local address 127.0.0.1:49912)
0811 22:09:39.430497 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:39.430580 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:33523: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260811 22:09:39.430862 1899 catalog_manager.cc:4584] TS 4f79e9ce91aa4402af85747fc3451c26 (127.28.47.129:33523): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 37903598fb634169bfce4b560e6921e2: Network error: Client connection negotiation failed: client connection to 127.28.47.129:33523: connect: Connection refused (error 111)
I20260811 22:09:39.437377 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 2300
I20260811 22:09:39.447489 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 1878
2026-08-11T22:09:39Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1262 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260811 22:09:39.473685 28862 mini_oidc.cc:152] Starting JWKS server
I20260811 22:09:39.475626 28862 webserver.cc:460] Webserver started at https://127.0.0.1:42883/ using document root <none> and password file <none>
I20260811 22:09:39.475842 28862 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:09:39.476147 28862 webserver.cc:460] Webserver started at http://127.0.0.1:38233/ using document root <none> and password file <none>
2026-08-11T22:09:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:39Z Disabled control of system clock
I20260811 22:09:39.487408 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:46873
--webserver_interface=127.28.47.190
--webserver_port=0
--builtin_ntp_servers=127.28.47.148:45243
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:46873
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42883/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/testchainca.cert with env {}
W20260811 22:09:39.607337 2514 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:39.607744 2514 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:39.607846 2514 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:39.612488 2514 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:39.612594 2514 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:39.612627 2514 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:39.612669 2514 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:39.612701 2514 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:09:39.612740 2514 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:09:39.612769 2514 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42883/jwks.json
I20260811 22:09:39.618501 2514 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:45243
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:46873
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:46873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42883/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2514
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:39.619751 2514 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:39.620841 2514 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:39.630903 2522 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:39.630934 2520 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:39.630924 2519 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:39.631383 2514 server_base.cc:1034] running on GCE node
I20260811 22:09:39.632063 2514 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:39.633376 2514 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:39.634574 2514 hybrid_clock.cc:648] HybridClock initialized: now 1786486179634516 us; error 71 us; skew 500 ppm
I20260811 22:09:39.637401 2514 webserver.cc:460] Webserver started at http://127.28.47.190:44805/ using document root <none> and password file <none>
I20260811 22:09:39.638114 2514 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:39.638192 2514 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:39.638394 2514 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:39.640933 2514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "8046adbc166f4db6bf78773e5847f9cf"
format_stamp: "Formatted at 2026-08-11 22:09:39 on dist-test-slave-vxj2"
server_key: "56a9213025bc3f814c275883be3dfb8f"
server_key_iv: "b93c477b0a39acec2e10d75f412c4ecc"
server_key_version: "encryptionkey@0"
I20260811 22:09:39.641703 2514 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "8046adbc166f4db6bf78773e5847f9cf"
format_stamp: "Formatted at 2026-08-11 22:09:39 on dist-test-slave-vxj2"
server_key: "56a9213025bc3f814c275883be3dfb8f"
server_key_iv: "b93c477b0a39acec2e10d75f412c4ecc"
server_key_version: "encryptionkey@0"
I20260811 22:09:39.646193 2514 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:09:39.649000 2528 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:39.650470 2514 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:39.650607 2514 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "8046adbc166f4db6bf78773e5847f9cf"
format_stamp: "Formatted at 2026-08-11 22:09:39 on dist-test-slave-vxj2"
server_key: "56a9213025bc3f814c275883be3dfb8f"
server_key_iv: "b93c477b0a39acec2e10d75f412c4ecc"
server_key_version: "encryptionkey@0"
I20260811 22:09:39.650717 2514 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:39.678189 2514 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:39.678932 2514 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:39.679107 2514 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:39.686393 2514 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:46873
I20260811 22:09:39.686414 2586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:46873 every 8 connection(s)
I20260811 22:09:39.687445 2514 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:39.690868 2587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:39.695241 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 2514
I20260811 22:09:39.695353 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:39.695653 28862 external_mini_cluster.cc:1428] Setting key 7c830b1a0f9615ab660d72a99417d1a5
I20260811 22:09:39.697396 2587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf: Bootstrap starting.
I20260811 22:09:39.700568 2587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:39.701583 2587 log.cc:824] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:39.704272 2587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf: No bootstrap required, opened a new log
I20260811 22:09:39.707448 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.697326 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40274 (local address 127.28.47.190:46873)
0811 22:09:39.698068 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:39.698089 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:39.698134 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:09:39.699154 (+ 1020us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:39.699179 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:39.699702 (+ 523us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:39.700077 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:39.700576 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:39.702127 (+ 1551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:39.703259 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:39.703968 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:39.704149 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:39.705078 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:39.705084 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:39.705102 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:39.705678 (+ 576us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:39.705756 (+ 78us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:39.705783 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:39.705892 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:39.706074 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 22:09:39.706395 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":140,"threads_started":1}
I20260811 22:09:39.709012 2587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8046adbc166f4db6bf78773e5847f9cf" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46873 } }
I20260811 22:09:39.709313 2587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:39.709393 2587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8046adbc166f4db6bf78773e5847f9cf, State: Initialized, Role: FOLLOWER
I20260811 22:09:39.709852 2587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8046adbc166f4db6bf78773e5847f9cf" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46873 } }
I20260811 22:09:39.710033 2587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:39.710109 2587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:39.710225 2587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:39.710646 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:09:39.711395 2587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8046adbc166f4db6bf78773e5847f9cf" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46873 } }
I20260811 22:09:39.711840 2587 leader_election.cc:302] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8046adbc166f4db6bf78773e5847f9cf; no voters:
I20260811 22:09:39.712230 2587 leader_election.cc:288] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:39.712328 2595 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:39.712558 2595 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [term 1 LEADER]: Becoming Leader. State: Replica: 8046adbc166f4db6bf78773e5847f9cf, State: Running, Role: LEADER
I20260811 22:09:39.712893 2595 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8046adbc166f4db6bf78773e5847f9cf" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46873 } }
I20260811 22:09:39.713703 2587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:39.714222 2597 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8046adbc166f4db6bf78773e5847f9cf. Latest consensus state: current_term: 1 leader_uuid: "8046adbc166f4db6bf78773e5847f9cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8046adbc166f4db6bf78773e5847f9cf" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46873 } } }
I20260811 22:09:39.714484 2597 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:39.715245 2596 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8046adbc166f4db6bf78773e5847f9cf" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8046adbc166f4db6bf78773e5847f9cf" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46873 } } }
I20260811 22:09:39.715399 2596 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:39.715549 2603 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:39.719484 2603 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:39.726138 2603 catalog_manager.cc:1347] Generated new cluster ID: adb5cebbe600466888e70c5d15dae285
I20260811 22:09:39.726276 2603 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:39.739552 2603 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:39.740789 2603 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:39.745894 2603 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8046adbc166f4db6bf78773e5847f9cf: Generated new TSK 0
I20260811 22:09:39.746845 2603 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:09:39.782387 2628 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260811 22:09:39.782610 2628 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:09:39.783167 2591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:39.765209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40282 (local address 127.28.47.190:46873)
0811 22:09:39.765374 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:39.765378 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:39.765653 (+ 275us) server_negotiation.cc:407] Connection header received
0811 22:09:39.765916 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:39.765919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:39.765976 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:39.766071 (+ 95us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:09:39.766389 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:39.766990 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:39.768238 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:39.768479 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:39.768567 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:39.769032 (+ 465us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:09:39.782770 (+ 13738us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42883/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:09:39.783051 (+ 281us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:40282: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42883/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":63}
W20260811 22:09:39.783279 2591 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:40282: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42883/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260811 22:09:39.785099 2627 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:39.765064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:46873 (local address 127.0.0.1:40282)
0811 22:09:39.765506 (+ 442us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:39.765522 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:39.765685 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:39.766112 (+ 427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:39.766116 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:39.766126 (+ 10us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:09:39.766274 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:39.766281 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:39.767108 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:39.767112 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:39.768102 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:39.768114 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:39.768606 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:39.768610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:39.768708 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:39.768777 (+ 69us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:09:39.783056 (+ 14279us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42883/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0811 22:09:39.783365 (+ 309us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.47.190:46873: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42883/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":164,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.28.47.190:46873: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42883/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260811 22:09:39.786242 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 2514
2026-08-11T22:09:39Z chronyd exiting
I20260811 22:09:39.827231 28862 test_util.cc:175] -----------------------------------------------
I20260811 22:09:39.827358 28862 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786486147647525-28862-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (355 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260811 22:09:39.828802 28862 mini_oidc.cc:152] Starting JWKS server
I20260811 22:09:39.829900 28862 webserver.cc:460] Webserver started at https://127.0.0.1:35215/ using document root <none> and password file <none>
I20260811 22:09:39.830049 28862 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:09:39.830370 28862 webserver.cc:460] Webserver started at http://127.0.0.1:36963/ using document root <none> and password file <none>
2026-08-11T22:09:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:39Z Disabled control of system clock
I20260811 22:09:39.841008 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:40005
--webserver_interface=127.28.47.190
--webserver_port=0
--builtin_ntp_servers=127.28.47.148:46375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:40005
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35215/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/testchainca.cert with env {}
W20260811 22:09:39.968554 2636 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:39.968884 2636 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:39.968931 2636 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:39.973203 2636 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:39.973284 2636 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:39.973300 2636 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:39.973320 2636 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:39.973335 2636 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:09:39.973352 2636 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:09:39.973366 2636 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:35215/jwks.json
I20260811 22:09:39.979393 2636 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:46375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:40005
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:40005
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:35215/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2636
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:39.980589 2636 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:39.981653 2636 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:39.990803 2645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:39.990803 2642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:39.990852 2643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:39.991282 2636 server_base.cc:1034] running on GCE node
I20260811 22:09:39.991916 2636 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:39.993274 2636 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:39.994443 2636 hybrid_clock.cc:648] HybridClock initialized: now 1786486179994430 us; error 42 us; skew 500 ppm
I20260811 22:09:39.996548 2636 webserver.cc:460] Webserver started at http://127.28.47.190:33803/ using document root <none> and password file <none>
I20260811 22:09:39.997291 2636 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:39.997372 2636 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:39.997607 2636 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:40.000046 2636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "ec1b0121fea2444d8a6aee2076e72a0b"
format_stamp: "Formatted at 2026-08-11 22:09:39 on dist-test-slave-vxj2"
server_key: "251f667cade07fd0eee0ebd3fdf58165"
server_key_iv: "cb87f1493ca48c69d807190ed16dc7f9"
server_key_version: "encryptionkey@0"
I20260811 22:09:40.000720 2636 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "ec1b0121fea2444d8a6aee2076e72a0b"
format_stamp: "Formatted at 2026-08-11 22:09:39 on dist-test-slave-vxj2"
server_key: "251f667cade07fd0eee0ebd3fdf58165"
server_key_iv: "cb87f1493ca48c69d807190ed16dc7f9"
server_key_version: "encryptionkey@0"
I20260811 22:09:40.005949 2636 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:09:40.010141 2652 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:40.012121 2636 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:09:40.012349 2636 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "ec1b0121fea2444d8a6aee2076e72a0b"
format_stamp: "Formatted at 2026-08-11 22:09:39 on dist-test-slave-vxj2"
server_key: "251f667cade07fd0eee0ebd3fdf58165"
server_key_iv: "cb87f1493ca48c69d807190ed16dc7f9"
server_key_version: "encryptionkey@0"
I20260811 22:09:40.012526 2636 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:40.036166 2636 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:40.037021 2636 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:40.037295 2636 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:40.045086 2636 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:40005
I20260811 22:09:40.045143 2714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:40005 every 8 connection(s)
I20260811 22:09:40.046249 2636 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:40.048848 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 2636
I20260811 22:09:40.049002 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:40.049402 28862 external_mini_cluster.cc:1428] Setting key 0f354c5687ca55fac4cac1f9d7dfab4f
I20260811 22:09:40.051486 2715 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:40.058195 2715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b: Bootstrap starting.
I20260811 22:09:40.060421 2719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:40.051050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33314 (local address 127.28.47.190:40005)
0811 22:09:40.051935 (+ 885us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:40.051954 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:40.051995 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:09:40.052760 (+ 765us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:40.052779 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:40.053315 (+ 536us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:40.053693 (+ 378us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:40.054179 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.055203 (+ 1024us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.056200 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.056925 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.057137 (+ 212us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:40.058387 (+ 1250us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:40.058394 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:40.058410 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:40.058866 (+ 456us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:40.058936 (+ 70us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:40.058961 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:40.059080 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:40.059247 (+ 167us) server_negotiation.cc:299] Negotiation successful
0811 22:09:40.059586 (+ 339us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":438,"thread_start_us":218,"threads_started":1}
I20260811 22:09:40.061014 2715 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:40.061874 2715 log.cc:824] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:40.063892 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:09:40.064314 2715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b: No bootstrap required, opened a new log
I20260811 22:09:40.068368 2715 raft_consensus.cc:348] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1b0121fea2444d8a6aee2076e72a0b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 40005 } }
I20260811 22:09:40.068650 2715 raft_consensus.cc:374] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:40.068687 2715 raft_consensus.cc:729] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ec1b0121fea2444d8a6aee2076e72a0b, State: Initialized, Role: FOLLOWER
I20260811 22:09:40.069247 2715 consensus_queue.cc:260] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1b0121fea2444d8a6aee2076e72a0b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 40005 } }
I20260811 22:09:40.069404 2715 raft_consensus.cc:388] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:40.069510 2715 raft_consensus.cc:482] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:40.069628 2715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:40.070763 2715 raft_consensus.cc:504] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1b0121fea2444d8a6aee2076e72a0b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 40005 } }
I20260811 22:09:40.071187 2715 leader_election.cc:302] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ec1b0121fea2444d8a6aee2076e72a0b; no voters:
I20260811 22:09:40.071593 2715 leader_election.cc:288] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:40.071750 2721 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:40.072026 2721 raft_consensus.cc:686] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [term 1 LEADER]: Becoming Leader. State: Replica: ec1b0121fea2444d8a6aee2076e72a0b, State: Running, Role: LEADER
I20260811 22:09:40.072383 2721 consensus_queue.cc:237] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1b0121fea2444d8a6aee2076e72a0b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 40005 } }
I20260811 22:09:40.073014 2715 sys_catalog.cc:564] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:40.074338 2722 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ec1b0121fea2444d8a6aee2076e72a0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1b0121fea2444d8a6aee2076e72a0b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 40005 } } }
I20260811 22:09:40.074355 2723 sys_catalog.cc:455] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader ec1b0121fea2444d8a6aee2076e72a0b. Latest consensus state: current_term: 1 leader_uuid: "ec1b0121fea2444d8a6aee2076e72a0b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ec1b0121fea2444d8a6aee2076e72a0b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 40005 } } }
I20260811 22:09:40.074501 2722 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:40.074501 2723 sys_catalog.cc:458] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b [sys.catalog]: This master's current role is: LEADER
W20260811 22:09:40.078953 2737 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:09:40.079082 2737 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:09:40.079154 2731 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:40.080040 2731 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:40.087188 2731 catalog_manager.cc:1347] Generated new cluster ID: 18a8924cd7084383be0a6b349778e329
I20260811 22:09:40.087306 2731 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:40.095793 2731 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:40.096896 2731 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:40.105470 2731 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ec1b0121fea2444d8a6aee2076e72a0b: Generated new TSK 0
I20260811 22:09:40.106518 2731 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260811 22:09:40.143559 2749 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260811 22:09:40.143664 2749 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260811 22:09:40.144500 2719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:40.120506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33318 (local address 127.28.47.190:40005)
0811 22:09:40.120644 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:40.120649 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:40.120897 (+ 248us) server_negotiation.cc:407] Connection header received
0811 22:09:40.121124 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:40.121127 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:40.121232 (+ 105us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:40.121359 (+ 127us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:09:40.121630 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.122299 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.123258 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.123498 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.123614 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:40.123971 (+ 357us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:09:40.143958 (+ 19987us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35215/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:09:40.144300 (+ 342us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33318: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35215/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 22:09:40.144563 2748 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:40.120399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:40005 (local address 127.0.0.1:33318)
0811 22:09:40.120752 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:40.120765 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:40.120967 (+ 202us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:40.121358 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:40.121361 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:40.121369 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:09:40.121500 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:40.121512 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.122439 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.122442 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:40.123128 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:40.123138 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.123627 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.123632 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:40.123740 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:40.123768 (+ 28us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:09:40.144197 (+ 20429us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35215/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0811 22:09:40.144387 (+ 190us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.47.190:40005: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35215/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":267,"thread_start_us":123,"threads_started":1}
W20260811 22:09:40.144588 2719 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33318: Failed to load JWKS: Error downloading JWKS from 'https://localhost:35215/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
I20260811 22:09:40.145326 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 2636
2026-08-11T22:09:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (360 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260811 22:09:40.189234 28862 mini_oidc.cc:152] Starting JWKS server
I20260811 22:09:40.190168 28862 webserver.cc:460] Webserver started at https://127.0.0.1:41977/ using document root <none> and password file <none>
I20260811 22:09:40.190306 28862 mini_oidc.cc:189] Starting OIDC Discovery server
I20260811 22:09:40.190584 28862 webserver.cc:460] Webserver started at http://127.0.0.1:45449/ using document root <none> and password file <none>
2026-08-11T22:09:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:40Z Disabled control of system clock
I20260811 22:09:40.202484 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45861
--webserver_interface=127.28.47.190
--webserver_port=0
--builtin_ntp_servers=127.28.47.148:43969
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:45861
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41977/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:09:40.322036 2756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:40.322360 2756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:40.322402 2756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:40.326565 2756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:40.326642 2756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:40.326658 2756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:40.326675 2756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:40.326691 2756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:09:40.326707 2756 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:09:40.326721 2756 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:41977/jwks.json
I20260811 22:09:40.331597 2756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:43969
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:45861
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:41977/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:40.332746 2756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:40.334261 2756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:40.343870 2764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:40.343842 2761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:40.344028 2756 server_base.cc:1034] running on GCE node
W20260811 22:09:40.343843 2762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:40.344831 2756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:40.346154 2756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:40.347355 2756 hybrid_clock.cc:648] HybridClock initialized: now 1786486180347337 us; error 34 us; skew 500 ppm
I20260811 22:09:40.349727 2756 webserver.cc:460] Webserver started at http://127.28.47.190:36927/ using document root <none> and password file <none>
I20260811 22:09:40.350308 2756 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:40.350390 2756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:40.350572 2756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:40.352777 2756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "7dbcf2b63172425cb4724ef0425ad6d9"
format_stamp: "Formatted at 2026-08-11 22:09:40 on dist-test-slave-vxj2"
server_key: "405c2e190a2495dcac1b8dac0a125208"
server_key_iv: "8c3bba9a13f89c002c7a3ca4ce18c2dd"
server_key_version: "encryptionkey@0"
I20260811 22:09:40.353466 2756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "7dbcf2b63172425cb4724ef0425ad6d9"
format_stamp: "Formatted at 2026-08-11 22:09:40 on dist-test-slave-vxj2"
server_key: "405c2e190a2495dcac1b8dac0a125208"
server_key_iv: "8c3bba9a13f89c002c7a3ca4ce18c2dd"
server_key_version: "encryptionkey@0"
I20260811 22:09:40.357932 2756 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:09:40.360754 2770 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:40.362354 2756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:40.362521 2756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "7dbcf2b63172425cb4724ef0425ad6d9"
format_stamp: "Formatted at 2026-08-11 22:09:40 on dist-test-slave-vxj2"
server_key: "405c2e190a2495dcac1b8dac0a125208"
server_key_iv: "8c3bba9a13f89c002c7a3ca4ce18c2dd"
server_key_version: "encryptionkey@0"
I20260811 22:09:40.362655 2756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:40.377614 2756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:40.378470 2756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:40.378702 2756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:40.387025 2756 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:45861
I20260811 22:09:40.387123 2832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:45861 every 8 connection(s)
I20260811 22:09:40.388202 2756 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:40.389907 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 2756
I20260811 22:09:40.390061 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:40.390359 28862 external_mini_cluster.cc:1428] Setting key 6a760433200ebff68631a78620387822
I20260811 22:09:40.392534 2833 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:40.400188 2833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9: Bootstrap starting.
I20260811 22:09:40.401371 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:40.391927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33530 (local address 127.28.47.190:45861)
0811 22:09:40.392861 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:40.392883 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:40.392926 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:09:40.393791 (+ 865us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:40.393813 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:40.394376 (+ 563us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:40.394705 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:40.395077 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.396075 (+ 998us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.397407 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.397991 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.398143 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:40.399167 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:40.399173 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:40.399188 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:40.399732 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:40.399826 (+ 94us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:40.399853 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:40.399955 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:40.400123 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 22:09:40.400483 (+ 360us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":480,"thread_start_us":199,"threads_started":1}
I20260811 22:09:40.402902 2833 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:40.403795 2833 log.cc:824] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:40.404495 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:09:40.406503 2833 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9: No bootstrap required, opened a new log
I20260811 22:09:40.410466 2833 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbcf2b63172425cb4724ef0425ad6d9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45861 } }
I20260811 22:09:40.410756 2833 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:40.410817 2833 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7dbcf2b63172425cb4724ef0425ad6d9, State: Initialized, Role: FOLLOWER
I20260811 22:09:40.411309 2833 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbcf2b63172425cb4724ef0425ad6d9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45861 } }
I20260811 22:09:40.411442 2833 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:40.411528 2833 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:40.411646 2833 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:40.412753 2833 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbcf2b63172425cb4724ef0425ad6d9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45861 } }
I20260811 22:09:40.413136 2833 leader_election.cc:302] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7dbcf2b63172425cb4724ef0425ad6d9; no voters:
I20260811 22:09:40.413532 2833 leader_election.cc:288] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:40.413717 2839 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:40.413980 2839 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [term 1 LEADER]: Becoming Leader. State: Replica: 7dbcf2b63172425cb4724ef0425ad6d9, State: Running, Role: LEADER
I20260811 22:09:40.414330 2839 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbcf2b63172425cb4724ef0425ad6d9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45861 } }
I20260811 22:09:40.415037 2833 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:40.416258 2842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7dbcf2b63172425cb4724ef0425ad6d9. Latest consensus state: current_term: 1 leader_uuid: "7dbcf2b63172425cb4724ef0425ad6d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbcf2b63172425cb4724ef0425ad6d9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45861 } } }
I20260811 22:09:40.416406 2842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:40.416712 2840 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7dbcf2b63172425cb4724ef0425ad6d9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7dbcf2b63172425cb4724ef0425ad6d9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45861 } } }
I20260811 22:09:40.416818 2840 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9 [sys.catalog]: This master's current role is: LEADER
W20260811 22:09:40.420781 2857 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:09:40.420879 2857 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:09:40.420965 2849 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:40.421759 2849 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:40.428038 2849 catalog_manager.cc:1347] Generated new cluster ID: ea16f73aec22412a84bded15309bbceb
I20260811 22:09:40.428135 2849 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:40.451236 2849 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:40.452359 2849 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:40.462067 2849 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7dbcf2b63172425cb4724ef0425ad6d9: Generated new TSK 0
I20260811 22:09:40.462925 2849 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:09:40.485973 2837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:40.479675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33544 (local address 127.28.47.190:45861)
0811 22:09:40.479884 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:40.479892 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:40.480222 (+ 330us) server_negotiation.cc:407] Connection header received
0811 22:09:40.480436 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:40.480439 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:40.480504 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:40.480614 (+ 110us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:09:40.480918 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.481590 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.482957 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.483230 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.483340 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:40.483761 (+ 421us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:09:40.485504 (+ 1743us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41977/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:09:40.485797 (+ 293us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33544: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41977/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 22:09:40.486066 2837 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33544: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41977/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260811 22:09:40.487811 2866 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:40.479566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45861 (local address 127.0.0.1:33544)
0811 22:09:40.480101 (+ 535us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:40.480124 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:40.480255 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:40.480641 (+ 386us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:40.480645 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:40.480653 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:09:40.480781 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:40.480788 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.481718 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.481722 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:40.482789 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:40.482805 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.483384 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.483388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:40.483504 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:40.483541 (+ 37us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:09:40.485714 (+ 2173us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41977/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0811 22:09:40.487622 (+ 1908us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.47.190:45861: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:41977/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":1656,"thread_start_us":265,"threads_started":1}
I20260811 22:09:40.489045 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 2756
E20260811 22:09:40.500953 2868 webserver.cc:533] Webserver: sslize failed:
E20260811 22:09:40.501050 2868 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-11T22:09:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (344 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-11T22:09:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:40Z Disabled control of system clock
I20260811 22:09:40.544687 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45065
--webserver_interface=127.28.47.190
--webserver_port=0
--builtin_ntp_servers=127.28.47.148:44445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:45065
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/testchainca.cert with env {}
W20260811 22:09:40.668546 2874 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:40.668879 2874 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:40.668926 2874 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:40.673269 2874 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:40.673345 2874 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:40.673362 2874 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:40.673380 2874 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:40.673396 2874 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:09:40.673413 2874 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260811 22:09:40.673427 2874 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260811 22:09:40.679356 2874 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:44445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:45065
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2874
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:40.680572 2874 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:40.681646 2874 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:40.691354 2882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:40.691359 2879 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:40.691354 2880 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:40.691620 2874 server_base.cc:1034] running on GCE node
I20260811 22:09:40.692493 2874 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:40.693883 2874 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:40.695080 2874 hybrid_clock.cc:648] HybridClock initialized: now 1786486180695045 us; error 52 us; skew 500 ppm
I20260811 22:09:40.697645 2874 webserver.cc:460] Webserver started at http://127.28.47.190:38577/ using document root <none> and password file <none>
I20260811 22:09:40.698482 2874 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:40.698604 2874 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:40.698886 2874 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:40.701490 2874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "7d6931ffecbc47889cac1f36143c20b3"
format_stamp: "Formatted at 2026-08-11 22:09:40 on dist-test-slave-vxj2"
server_key: "c26302b61d1d53ee0a1bbda70f8b7b18"
server_key_iv: "49ff6c8ca1dd11ea1794b430cb8df34d"
server_key_version: "encryptionkey@0"
I20260811 22:09:40.702154 2874 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "7d6931ffecbc47889cac1f36143c20b3"
format_stamp: "Formatted at 2026-08-11 22:09:40 on dist-test-slave-vxj2"
server_key: "c26302b61d1d53ee0a1bbda70f8b7b18"
server_key_iv: "49ff6c8ca1dd11ea1794b430cb8df34d"
server_key_version: "encryptionkey@0"
I20260811 22:09:40.706463 2874 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 22:09:40.709264 2889 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:40.710696 2874 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:09:40.710844 2874 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "7d6931ffecbc47889cac1f36143c20b3"
format_stamp: "Formatted at 2026-08-11 22:09:40 on dist-test-slave-vxj2"
server_key: "c26302b61d1d53ee0a1bbda70f8b7b18"
server_key_iv: "49ff6c8ca1dd11ea1794b430cb8df34d"
server_key_version: "encryptionkey@0"
I20260811 22:09:40.710971 2874 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:40.725641 2874 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:40.726488 2874 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:40.726707 2874 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:40.734761 2874 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:45065
I20260811 22:09:40.734784 2951 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:45065 every 8 connection(s)
I20260811 22:09:40.735833 2874 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:40.739089 2952 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:40.742254 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 2874
I20260811 22:09:40.742410 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:40.742754 28862 external_mini_cluster.cc:1428] Setting key e849289c373779c42031978d25a15132
I20260811 22:09:40.746507 2952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3: Bootstrap starting.
I20260811 22:09:40.749043 2952 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:40.749897 2952 log.cc:824] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:40.752092 2952 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3: No bootstrap required, opened a new log
I20260811 22:09:40.752820 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:40.744283 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53566 (local address 127.28.47.190:45065)
0811 22:09:40.745121 (+ 838us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:40.745139 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:40.745271 (+ 132us) server_negotiation.cc:407] Connection header received
0811 22:09:40.745961 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:40.745979 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:40.746480 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:40.746871 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:40.747355 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.748375 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.749394 (+ 1019us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.749911 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.750105 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:40.750993 (+ 888us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:40.750999 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:40.751014 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:09:40.751426 (+ 412us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:40.751485 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:09:40.751505 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:40.751619 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:40.751750 (+ 131us) server_negotiation.cc:299] Negotiation successful
0811 22:09:40.752025 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":170,"threads_started":1}
I20260811 22:09:40.756196 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:09:40.756618 2952 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6931ffecbc47889cac1f36143c20b3" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45065 } }
I20260811 22:09:40.756911 2952 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:40.756989 2952 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6931ffecbc47889cac1f36143c20b3, State: Initialized, Role: FOLLOWER
I20260811 22:09:40.757575 2952 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6931ffecbc47889cac1f36143c20b3" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45065 } }
I20260811 22:09:40.757748 2952 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:40.757829 2952 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:40.757934 2952 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:40.759068 2952 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6931ffecbc47889cac1f36143c20b3" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45065 } }
I20260811 22:09:40.759465 2952 leader_election.cc:302] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d6931ffecbc47889cac1f36143c20b3; no voters:
I20260811 22:09:40.759833 2952 leader_election.cc:288] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:40.759979 2958 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:40.760282 2958 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [term 1 LEADER]: Becoming Leader. State: Replica: 7d6931ffecbc47889cac1f36143c20b3, State: Running, Role: LEADER
I20260811 22:09:40.760620 2958 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6931ffecbc47889cac1f36143c20b3" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45065 } }
I20260811 22:09:40.761284 2952 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:40.762509 2960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d6931ffecbc47889cac1f36143c20b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6931ffecbc47889cac1f36143c20b3" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45065 } } }
I20260811 22:09:40.762694 2960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:40.763082 2961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d6931ffecbc47889cac1f36143c20b3. Latest consensus state: current_term: 1 leader_uuid: "7d6931ffecbc47889cac1f36143c20b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6931ffecbc47889cac1f36143c20b3" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45065 } } }
I20260811 22:09:40.763163 2969 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:40.763208 2961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:40.766459 2969 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:40.773204 2969 catalog_manager.cc:1347] Generated new cluster ID: 0082bc6d97044510a46d167413c3ae38
I20260811 22:09:40.773310 2969 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:40.783927 2969 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:40.784983 2969 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:40.797331 2969 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d6931ffecbc47889cac1f36143c20b3: Generated new TSK 0
I20260811 22:09:40.798135 2969 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:09:40.851730 2956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:40.813801 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53568 (local address 127.28.47.190:45065)
0811 22:09:40.813953 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:40.813959 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:40.814166 (+ 207us) server_negotiation.cc:407] Connection header received
0811 22:09:40.814381 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:40.814384 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:40.814445 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:40.814566 (+ 121us) server_negotiation.cc:226] Negotiated authn=JWT
0811 22:09:40.814849 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.815486 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.816502 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.816739 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.816854 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:40.817256 (+ 402us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0811 22:09:40.851093 (+ 33837us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:09:40.851521 (+ 428us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:53568: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 22:09:40.851840 2956 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:53568: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260811 22:09:40.851928 2988 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:40.813683 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45065 (local address 127.0.0.1:53568)
0811 22:09:40.814070 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:40.814082 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:40.814225 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:40.814562 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:40.814565 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:40.814573 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0811 22:09:40.814695 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:40.814701 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.815625 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.815629 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:40.816372 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:40.816381 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:40.816872 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:40.816876 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:40.816986 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:40.817013 (+ 27us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0811 22:09:40.851388 (+ 34375us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0811 22:09:40.851700 (+ 312us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.28.47.190:45065: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":124,"threads_started":1}
I20260811 22:09:40.852370 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 2874
2026-08-11T22:09:40Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (341 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (137 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (138 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:41 dist-test-slave-vxj2 krb5kdc[3002](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:41 dist-test-slave-vxj2 krb5kdc[3002](info): set up 2 sockets
Aug 11 22:09:41 dist-test-slave-vxj2 krb5kdc[3002](info): commencing operation
krb5kdc: starting...
W20260811 22:09:42.325455 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.173s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:09:42 dist-test-slave-vxj2 krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:09:42.470136 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45649
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:35823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:45649
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:42.590945 3017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:42.591266 3017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:42.591308 3017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:42.595585 3017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:42.595670 3017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:42.595690 3017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:42.595709 3017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:42.595726 3017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:42.601459 3017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:35823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:45649
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45649
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:42.602619 3017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:42.603686 3017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:42.613121 3027 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:42.613124 3024 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:42.613250 3017 server_base.cc:1034] running on GCE node
W20260811 22:09:42.613124 3023 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:42.614118 3017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:42.615350 3017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:42.616583 3017 hybrid_clock.cc:648] HybridClock initialized: now 1786486182616526 us; error 67 us; skew 500 ppm
Aug 11 22:09:42 dist-test-slave-vxj2 krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486182, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:42.619874 3017 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:09:42.620918 3017 webserver.cc:460] Webserver started at http://127.28.47.190:42401/ using document root <none> and password file <none>
I20260811 22:09:42.621593 3017 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:42.621697 3017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:42.621968 3017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:42.624337 3017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "fcc86c677bca4daf89b9105168afaac6"
format_stamp: "Formatted at 2026-08-11 22:09:42 on dist-test-slave-vxj2"
server_key: "4017cc9376ed53f9ad570d6b1e5aaff6"
server_key_iv: "4a18d7cf47d65a7b1bcbfe3cef7e2c24"
server_key_version: "encryptionkey@0"
I20260811 22:09:42.624970 3017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "fcc86c677bca4daf89b9105168afaac6"
format_stamp: "Formatted at 2026-08-11 22:09:42 on dist-test-slave-vxj2"
server_key: "4017cc9376ed53f9ad570d6b1e5aaff6"
server_key_iv: "4a18d7cf47d65a7b1bcbfe3cef7e2c24"
server_key_version: "encryptionkey@0"
I20260811 22:09:42.629127 3017 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:09:42.631948 3035 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:42.633387 3017 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:42.633543 3017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "fcc86c677bca4daf89b9105168afaac6"
format_stamp: "Formatted at 2026-08-11 22:09:42 on dist-test-slave-vxj2"
server_key: "4017cc9376ed53f9ad570d6b1e5aaff6"
server_key_iv: "4a18d7cf47d65a7b1bcbfe3cef7e2c24"
server_key_version: "encryptionkey@0"
I20260811 22:09:42.633673 3017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:42.656903 3017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:42.660606 3017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:42.660843 3017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:42.668704 3017 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:45649
I20260811 22:09:42.668807 3102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:45649 every 8 connection(s)
I20260811 22:09:42.669858 3017 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:42.673089 3103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:42.678587 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 3017
I20260811 22:09:42.678762 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:42.679054 28862 external_mini_cluster.cc:1428] Setting key 6a3de6b95cc779d3877d2741347085dc
I20260811 22:09:42.679632 3103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6: Bootstrap starting.
I20260811 22:09:42.682730 3103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:42.683913 3103 log.cc:824] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:42.686561 3103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6: No bootstrap required, opened a new log
Aug 11 22:09:42 dist-test-slave-vxj2 krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486182, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:42.691901 3103 raft_consensus.cc:348] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc86c677bca4daf89b9105168afaac6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45649 } }
I20260811 22:09:42.692268 3103 raft_consensus.cc:374] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:42.692328 3103 raft_consensus.cc:729] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fcc86c677bca4daf89b9105168afaac6, State: Initialized, Role: FOLLOWER
I20260811 22:09:42.692938 3103 consensus_queue.cc:260] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc86c677bca4daf89b9105168afaac6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45649 } }
I20260811 22:09:42.693084 3103 raft_consensus.cc:388] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:42.693197 3103 raft_consensus.cc:482] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:42.693329 3103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:42.694388 3103 raft_consensus.cc:504] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc86c677bca4daf89b9105168afaac6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45649 } }
I20260811 22:09:42.694789 3103 leader_election.cc:302] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fcc86c677bca4daf89b9105168afaac6; no voters:
I20260811 22:09:42.695184 3103 leader_election.cc:288] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:42.695350 3109 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:42.695581 3109 raft_consensus.cc:686] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [term 1 LEADER]: Becoming Leader. State: Replica: fcc86c677bca4daf89b9105168afaac6, State: Running, Role: LEADER
I20260811 22:09:42.695990 3109 consensus_queue.cc:237] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc86c677bca4daf89b9105168afaac6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45649 } }
I20260811 22:09:42.696411 3106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:42.680589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56406 (local address 127.28.47.190:45649)
0811 22:09:42.681080 (+ 491us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:42.681097 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:42.681136 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:09:42.681916 (+ 780us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:42.682010 (+ 94us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:42.682407 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:42.683082 (+ 675us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:42.684095 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:42.685229 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:42.686485 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:42.687081 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:42.687245 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:42.690388 (+ 3143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:42.690396 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:42.690420 (+ 24us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:42.690464 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:42.692656 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:42.693316 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:42.693324 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:42.693331 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:42.693510 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:42.693894 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:42.693898 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:42.693901 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:42.694393 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:42.694614 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:42.694948 (+ 334us) server_negotiation.cc:299] Negotiation successful
0811 22:09:42.695320 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":167,"threads_started":1}
I20260811 22:09:42.696712 3103 sys_catalog.cc:564] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:42.697398 3109 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fcc86c677bca4daf89b9105168afaac6. Latest consensus state: current_term: 1 leader_uuid: "fcc86c677bca4daf89b9105168afaac6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc86c677bca4daf89b9105168afaac6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45649 } } }
I20260811 22:09:42.697531 3109 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:42.698243 3110 sys_catalog.cc:455] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fcc86c677bca4daf89b9105168afaac6" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fcc86c677bca4daf89b9105168afaac6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45649 } } }
I20260811 22:09:42.698351 3110 sys_catalog.cc:458] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:42.699258 3116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:42.702468 3116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:42.709620 3116 catalog_manager.cc:1347] Generated new cluster ID: 89be31f8a145416087794c6b955c18b9
I20260811 22:09:42.709728 3116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:42.743209 3116 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:42.744319 3116 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:42.750308 3116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fcc86c677bca4daf89b9105168afaac6: Generated new TSK 0
I20260811 22:09:42.751194 3116 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:09:42.785416 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45649
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:35823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:42.911789 3135 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:42.912077 3135 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:42.912122 3135 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:42.916324 3135 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:42.916400 3135 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:42.916467 3135 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:42.922020 3135 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:35823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3135
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:42.923184 3135 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:42.924263 3135 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:42.934809 3140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:42.934871 3141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:42.934906 3145 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:42.935263 3135 server_base.cc:1034] running on GCE node
I20260811 22:09:42.935655 3135 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:42.936304 3135 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:42.937494 3135 hybrid_clock.cc:648] HybridClock initialized: now 1786486182937464 us; error 33 us; skew 500 ppm
Aug 11 22:09:42 dist-test-slave-vxj2 krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486182, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:42.940845 3135 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:09:42.941944 3135 webserver.cc:460] Webserver started at http://127.28.47.129:33261/ using document root <none> and password file <none>
I20260811 22:09:42.942575 3135 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:42.942673 3135 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:42.942971 3135 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:42.945360 3135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "7edf28218ebe4fdc9c70942d579edd69"
format_stamp: "Formatted at 2026-08-11 22:09:42 on dist-test-slave-vxj2"
server_key: "df170ee580e58685ae3d5a596b2d0c95"
server_key_iv: "c2f02fa74cea57640f4c4ab2ec622ee6"
server_key_version: "encryptionkey@0"
I20260811 22:09:42.946018 3135 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "7edf28218ebe4fdc9c70942d579edd69"
format_stamp: "Formatted at 2026-08-11 22:09:42 on dist-test-slave-vxj2"
server_key: "df170ee580e58685ae3d5a596b2d0c95"
server_key_iv: "c2f02fa74cea57640f4c4ab2ec622ee6"
server_key_version: "encryptionkey@0"
I20260811 22:09:42.950671 3135 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 22:09:42.953951 3154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:42.955427 3135 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:09:42.955605 3135 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "7edf28218ebe4fdc9c70942d579edd69"
format_stamp: "Formatted at 2026-08-11 22:09:42 on dist-test-slave-vxj2"
server_key: "df170ee580e58685ae3d5a596b2d0c95"
server_key_iv: "c2f02fa74cea57640f4c4ab2ec622ee6"
server_key_version: "encryptionkey@0"
I20260811 22:09:42.955749 3135 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:42.967983 3135 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:42.971776 3135 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:42.972052 3135 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:42.972781 3135 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:42.973889 3135 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:42.973963 3135 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:42.974042 3135 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:42.974087 3135 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:42.983691 3135 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:40319
I20260811 22:09:42.983800 3289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:40319 every 8 connection(s)
I20260811 22:09:42.984755 3135 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:42.994125 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 3135
I20260811 22:09:42.994274 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:09:42.994640 28862 external_mini_cluster.cc:1428] Setting key f53d24cfaacfacaf84177073410726bf
Aug 11 22:09:42 dist-test-slave-vxj2 krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486182, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:43.000976 3106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:42.986965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:39781 (local address 127.28.47.190:45649)
0811 22:09:42.987161 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:42.987165 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:42.988064 (+ 899us) server_negotiation.cc:407] Connection header received
0811 22:09:42.989620 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:42.989623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:42.989688 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:42.989804 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:42.991098 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:42.991655 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:42.992853 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:42.993104 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:42.993277 (+ 173us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:42.996445 (+ 3168us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:42.996448 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:42.996450 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:42.996478 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:42.998377 (+ 1899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:42.998981 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:42.998985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:42.998987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:42.999094 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:42.999552 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:42.999557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:42.999561 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:42.999870 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:42.999957 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:43.000605 (+ 648us) server_negotiation.cc:299] Negotiation successful
0811 22:09:43.000755 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:09:43.002046 3292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:42.987352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45649 (local address 127.28.47.129:39781)
0811 22:09:42.987896 (+ 544us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:42.987941 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:42.989392 (+ 1451us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:42.989998 (+ 606us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:42.990006 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:42.990401 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:42.990931 (+ 530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:42.990945 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:42.991800 (+ 855us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:42.991803 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:42.992707 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:42.992727 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:42.993343 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:42.993347 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:42.993528 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:42.994203 (+ 675us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:42.994256 (+ 53us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:42.996256 (+ 2000us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:42.998539 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:42.998546 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:42.998564 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:42.998877 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:42.999225 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:42.999234 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:42.999236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:42.999437 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:43.000006 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:43.000014 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:43.000324 (+ 310us) client_negotiation.cc:769] Sending connection context
0811 22:09:43.000559 (+ 235us) client_negotiation.cc:240] Negotiation successful
0811 22:09:43.000947 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":118,"threads_started":1}
I20260811 22:09:43.003721 3290 heartbeater.cc:347] Connected to a master server at 127.28.47.190:45649
I20260811 22:09:43.004076 3290 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:43.004675 3290 heartbeater.cc:511] Master 127.28.47.190:45649 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
I20260811 22:09:43.006793 3057 ts_manager.cc:194] Registered new tserver with Master: 7edf28218ebe4fdc9c70942d579edd69 (127.28.47.129:40319)
I20260811 22:09:43.008054 3057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:39781
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:09:43.028971 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45649
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:35823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:43.150102 3296 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:43.150435 3296 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:43.150511 3296 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:43.155921 3296 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:43.156031 3296 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:43.156136 3296 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:43.162245 3296 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:35823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3296
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:43.163643 3296 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:43.164767 3296 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:43.175622 3303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:43.175576 3301 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:43.175576 3305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:43.175809 3296 server_base.cc:1034] running on GCE node
I20260811 22:09:43.176462 3296 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:43.177284 3296 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:43.178494 3296 hybrid_clock.cc:648] HybridClock initialized: now 1786486183178468 us; error 35 us; skew 500 ppm
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:43.182236 3296 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:09:43.183318 3296 webserver.cc:460] Webserver started at http://127.28.47.130:43473/ using document root <none> and password file <none>
I20260811 22:09:43.183964 3296 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:43.184031 3296 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:43.184273 3296 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:43.186751 3296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "3afecf45e8a347aba50caadbd548d6c6"
format_stamp: "Formatted at 2026-08-11 22:09:43 on dist-test-slave-vxj2"
server_key: "7b4a3e6fbd7d8ca32f1f09f9c99f66bf"
server_key_iv: "c565dc6ea8db3b393622e43e0423aa85"
server_key_version: "encryptionkey@0"
I20260811 22:09:43.187409 3296 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "3afecf45e8a347aba50caadbd548d6c6"
format_stamp: "Formatted at 2026-08-11 22:09:43 on dist-test-slave-vxj2"
server_key: "7b4a3e6fbd7d8ca32f1f09f9c99f66bf"
server_key_iv: "c565dc6ea8db3b393622e43e0423aa85"
server_key_version: "encryptionkey@0"
I20260811 22:09:43.191994 3296 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:09:43.194981 3312 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:43.196693 3296 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:43.196868 3296 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "3afecf45e8a347aba50caadbd548d6c6"
format_stamp: "Formatted at 2026-08-11 22:09:43 on dist-test-slave-vxj2"
server_key: "7b4a3e6fbd7d8ca32f1f09f9c99f66bf"
server_key_iv: "c565dc6ea8db3b393622e43e0423aa85"
server_key_version: "encryptionkey@0"
I20260811 22:09:43.197014 3296 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:43.219391 3296 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:43.223232 3296 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:43.223474 3296 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:43.224136 3296 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:43.225067 3296 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:43.225155 3296 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:43.225252 3296 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:43.225309 3296 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:43.235121 3296 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:41623
I20260811 22:09:43.235145 3448 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:41623 every 8 connection(s)
I20260811 22:09:43.236137 3296 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:43.237529 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 3296
I20260811 22:09:43.237632 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:09:43.237999 28862 external_mini_cluster.cc:1428] Setting key 516014459757a689053523d3e3b54c95
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
I20260811 22:09:43.254254 3106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:43.238567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:34197 (local address 127.28.47.190:45649)
0811 22:09:43.238728 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:43.238733 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:43.239694 (+ 961us) server_negotiation.cc:407] Connection header received
0811 22:09:43.241259 (+ 1565us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:43.241262 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:43.241322 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:43.241404 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:43.242992 (+ 1588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.243548 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.245535 (+ 1987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.245799 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.245958 (+ 159us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:43.249346 (+ 3388us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:43.249351 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:43.249358 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:43.249402 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:43.251228 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:43.251949 (+ 721us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:43.251953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:43.251955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:43.252073 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:43.252502 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:43.252504 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:43.252505 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:43.252707 (+ 202us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:43.252800 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:43.253771 (+ 971us) server_negotiation.cc:299] Negotiation successful
0811 22:09:43.253971 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:09:43.255065 3451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:43.239014 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45649 (local address 127.28.47.130:34197)
0811 22:09:43.239530 (+ 516us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:43.239586 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:43.240985 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:43.241611 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:43.241619 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:43.242017 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:43.242770 (+ 753us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:43.242792 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.243696 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.243702 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:43.245329 (+ 1627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:43.245348 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.246049 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.246055 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:43.246319 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:43.247084 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:43.247103 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:43.249063 (+ 1960us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:43.251379 (+ 2316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:43.251386 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:43.251406 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:43.251735 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:43.252210 (+ 475us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:43.252213 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:43.252214 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:43.252400 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:43.253026 (+ 626us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:43.253036 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:43.253435 (+ 399us) client_negotiation.cc:769] Sending connection context
0811 22:09:43.253611 (+ 176us) client_negotiation.cc:240] Negotiation successful
0811 22:09:43.254048 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"thread_start_us":152,"threads_started":1}
I20260811 22:09:43.256636 3449 heartbeater.cc:347] Connected to a master server at 127.28.47.190:45649
I20260811 22:09:43.256999 3449 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:43.257781 3449 heartbeater.cc:511] Master 127.28.47.190:45649 requested a full tablet report, sending...
I20260811 22:09:43.259222 3057 ts_manager.cc:194] Registered new tserver with Master: 3afecf45e8a347aba50caadbd548d6c6 (127.28.47.130:41623)
I20260811 22:09:43.260161 3057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:34197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:09:43.273602 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45649
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:35823
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:43.397593 3454 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:43.397929 3454 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:43.398042 3454 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:43.402693 3454 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:43.402788 3454 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:43.402889 3454 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:43.408831 3454 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:35823
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45649
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3454
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:43.410148 3454 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:43.411278 3454 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:43.422207 3461 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:43.422220 3466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:43.422206 3462 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:43.422655 3454 server_base.cc:1034] running on GCE node
I20260811 22:09:43.423182 3454 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:43.423907 3454 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:43.425112 3454 hybrid_clock.cc:648] HybridClock initialized: now 1786486183425063 us; error 56 us; skew 500 ppm
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:43.428241 3454 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:09:43.429486 3454 webserver.cc:460] Webserver started at http://127.28.47.131:44185/ using document root <none> and password file <none>
I20260811 22:09:43.430114 3454 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:43.430223 3454 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:43.430497 3454 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:43.432788 3454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "8c78086435924f50956f7bb3b8fc4cd7"
format_stamp: "Formatted at 2026-08-11 22:09:43 on dist-test-slave-vxj2"
server_key: "684897fa2ff4dda3604457ead8117aea"
server_key_iv: "056b817d65faaaf837055e296301d43f"
server_key_version: "encryptionkey@0"
I20260811 22:09:43.433561 3454 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "8c78086435924f50956f7bb3b8fc4cd7"
format_stamp: "Formatted at 2026-08-11 22:09:43 on dist-test-slave-vxj2"
server_key: "684897fa2ff4dda3604457ead8117aea"
server_key_iv: "056b817d65faaaf837055e296301d43f"
server_key_version: "encryptionkey@0"
I20260811 22:09:43.442687 3454 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.004s sys 0.000s
I20260811 22:09:43.446053 3476 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:43.448005 3454 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:09:43.448164 3454 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "8c78086435924f50956f7bb3b8fc4cd7"
format_stamp: "Formatted at 2026-08-11 22:09:43 on dist-test-slave-vxj2"
server_key: "684897fa2ff4dda3604457ead8117aea"
server_key_iv: "056b817d65faaaf837055e296301d43f"
server_key_version: "encryptionkey@0"
I20260811 22:09:43.448289 3454 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:43.480091 3454 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:43.485028 3454 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:43.485307 3454 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:43.486387 3454 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:43.487488 3454 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:43.487552 3454 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:43.487597 3454 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:43.487613 3454 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:43.498122 3454 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:38723
I20260811 22:09:43.498142 3615 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:38723 every 8 connection(s)
I20260811 22:09:43.499781 3454 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:43.502372 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 3454
I20260811 22:09:43.502523 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:09:43.502853 28862 external_mini_cluster.cc:1428] Setting key 4262bdd005def7894a6e7dc0f23b50c0
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:43.517311 3106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:43.502894 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:54149 (local address 127.28.47.190:45649)
0811 22:09:43.503067 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:43.503072 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:43.504454 (+ 1382us) server_negotiation.cc:407] Connection header received
0811 22:09:43.506163 (+ 1709us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:43.506166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:43.506230 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:43.506311 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:43.507610 (+ 1299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.508247 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.509561 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.509806 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.509935 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:43.513019 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:43.513022 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:43.513024 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:43.513056 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:43.514741 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:43.515364 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:43.515367 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:43.515368 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:43.515452 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:43.515863 (+ 411us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:43.515867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:43.515868 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:43.516151 (+ 283us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:43.516256 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:43.516981 (+ 725us) server_negotiation.cc:299] Negotiation successful
0811 22:09:43.517146 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:09:43.518322 3620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:43.503438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45649 (local address 127.28.47.131:54149)
0811 22:09:43.504270 (+ 832us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:43.504338 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:43.505933 (+ 1595us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:43.506521 (+ 588us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:43.506530 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:43.506928 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:43.507453 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:43.507469 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.508392 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.508395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:43.509424 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:43.509436 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.509994 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.509998 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:43.510178 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:43.510962 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:43.510980 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:43.512789 (+ 1809us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:43.514936 (+ 2147us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:43.514943 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:43.514958 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:43.515258 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:43.515550 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:43.515555 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:43.515557 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:43.515737 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:43.516271 (+ 534us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:43.516283 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:43.516679 (+ 396us) client_negotiation.cc:769] Sending connection context
0811 22:09:43.516917 (+ 238us) client_negotiation.cc:240] Negotiation successful
0811 22:09:43.517368 (+ 451us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":556,"thread_start_us":203,"threads_started":1}
I20260811 22:09:43.519462 3617 heartbeater.cc:347] Connected to a master server at 127.28.47.190:45649
I20260811 22:09:43.519737 3617 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:43.520304 3617 heartbeater.cc:511] Master 127.28.47.190:45649 requested a full tablet report, sending...
I20260811 22:09:43.521569 3057 ts_manager.cc:194] Registered new tserver with Master: 8c78086435924f50956f7bb3b8fc4cd7 (127.28.47.131:38723)
I20260811 22:09:43.522236 3057 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:54149
I20260811 22:09:43.530184 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/test-admin.keytab.
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:43.551710 28862 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260811 22:09:43.555038 3627 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:43.553546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56130 (local address 127.28.47.129:40319)
0811 22:09:43.553915 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:43.553921 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:43.553949 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:09:43.554000 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:43.554005 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:43.554197 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:43.554332 (+ 135us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:43.554869 (+ 537us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56130: BlockingRecv error: recv got EOF from 127.0.0.1:56130 (error 108)
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":118,"threads_started":1}
W20260811 22:09:43.555459 28862 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:43.556645 28862 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:43.557761 28862 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.129@KRBTEST.COM
I20260811 22:09:43.569542 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:43.558825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56138 (local address 127.28.47.129:40319)
0811 22:09:43.558978 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:43.558982 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:43.558998 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:09:43.559185 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:43.559189 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:43.559240 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:43.559335 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:43.560094 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.561006 (+ 912us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.562013 (+ 1007us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.562460 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.562574 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:43.565372 (+ 2798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:43.565387 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:43.565403 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:43.565437 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:43.567284 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:43.567855 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:43.567862 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:43.567863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:43.567965 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:43.568332 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:43.568335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:43.568336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:43.568645 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:43.568967 (+ 322us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:43.569114 (+ 147us) server_negotiation.cc:299] Negotiation successful
0811 22:09:43.569390 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
W20260811 22:09:43.574227 3627 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:43.573086 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56152 (local address 127.28.47.129:40319)
0811 22:09:43.573261 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:43.573264 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:43.573374 (+ 110us) server_negotiation.cc:407] Connection header received
0811 22:09:43.573475 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:43.573478 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:43.573530 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:43.573608 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:43.574077 (+ 469us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56152: BlockingRecv error: recv got EOF from 127.0.0.1:56152 (error 108)
Metrics: {"server-negotiator.queue_time_us":89}
W20260811 22:09:43.574357 28862 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:43.575589 28862 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:43.576898 28862 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.129@KRBTEST.COM
I20260811 22:09:43.588973 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:43.578141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56158 (local address 127.28.47.129:40319)
0811 22:09:43.578344 (+ 203us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:43.578351 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:43.578512 (+ 161us) server_negotiation.cc:407] Connection header received
0811 22:09:43.578714 (+ 202us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:43.578720 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:43.578785 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:43.578887 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:43.579656 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.580237 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.581230 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.581501 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.581600 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:43.584666 (+ 3066us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:43.584672 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:43.584674 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:43.584720 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:43.586693 (+ 1973us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:43.587329 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:43.587334 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:43.587335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:43.587441 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:43.587831 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:43.587836 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:43.587838 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:43.588079 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:43.588290 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:43.588587 (+ 297us) server_negotiation.cc:299] Negotiation successful
0811 22:09:43.588750 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
W20260811 22:09:43.592384 3627 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:43.591215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56170 (local address 127.28.47.129:40319)
0811 22:09:43.591357 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:43.591361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:43.591492 (+ 131us) server_negotiation.cc:407] Connection header received
0811 22:09:43.591666 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:43.591669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:43.591719 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:43.591803 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:43.592261 (+ 458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56170: BlockingRecv error: recv got EOF from 127.0.0.1:56170 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260811 22:09:43.592731 28862 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786486147647525-28862-0/krb5kdc/krb5cc' of type FILE
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:43.594099 28862 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:43.595254 28862 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3002](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.129@KRBTEST.COM
I20260811 22:09:43.607342 3627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:43.597545 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56180 (local address 127.28.47.129:40319)
0811 22:09:43.597740 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:43.597745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:43.598027 (+ 282us) server_negotiation.cc:407] Connection header received
0811 22:09:43.598253 (+ 226us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:43.598257 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:43.598316 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:43.598423 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:43.599090 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.599622 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.600490 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:43.600748 (+ 258us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:43.600872 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:43.603856 (+ 2984us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:43.603860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:43.603862 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:43.603895 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:43.605581 (+ 1686us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:43.606112 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:43.606115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:43.606117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:43.606201 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:43.606542 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:43.606545 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:43.606546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:43.606736 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:43.606875 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:43.607098 (+ 223us) server_negotiation.cc:299] Negotiation successful
0811 22:09:43.607208 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:09:43.608742 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 3135
I20260811 22:09:43.619278 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 3296
I20260811 22:09:43.637311 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 3454
I20260811 22:09:43.646579 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 3017
2026-08-11T22:09:43Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2520 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3643](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3643](info): set up 2 sockets
Aug 11 22:09:43 dist-test-slave-vxj2 krb5kdc[3643](info): commencing operation
krb5kdc: starting...
W20260811 22:09:44.806128 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:09:44 dist-test-slave-vxj2 krb5kdc[3643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:44Z Disabled control of system clock
WARNING: no policy specified for oryx/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.190.keytab.
Entry for principal oryx/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.190.keytab.
I20260811 22:09:44.945634 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:42635
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.190.keytab
--principal=oryx/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:42635
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:45.067976 3658 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:45.068310 3658 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:45.068361 3658 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:45.072964 3658 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:45.073051 3658 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:45.073072 3658 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:45.073091 3658 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:45.073107 3658 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:45.079097 3658 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:42635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.190.keytab
--principal=oryx/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:42635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3658
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:45.080345 3658 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:45.081517 3658 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:45.091363 3664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:45.091477 3665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:45.091562 3658 server_base.cc:1034] running on GCE node
W20260811 22:09:45.091840 3667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:45.092687 3658 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:45.094120 3658 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:45.095291 3658 hybrid_clock.cc:648] HybridClock initialized: now 1786486185095255 us; error 55 us; skew 500 ppm
Aug 11 22:09:45 dist-test-slave-vxj2 krb5kdc[3643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486185, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:45.103223 3658 init.cc:377] Logged in from keytab as oryx/127.28.47.190@KRBTEST.COM (short username oryx)
I20260811 22:09:45.104367 3658 webserver.cc:460] Webserver started at http://127.28.47.190:36555/ using document root <none> and password file <none>
I20260811 22:09:45.104976 3658 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:45.105046 3658 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:45.105317 3658 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:45.107658 3658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "20c3d6dc17494bcab85cfa79c470a850"
server_key_iv: "7b29366673cc34b4b147502cca5d7b60"
server_key_version: "encryptionkey@0"
I20260811 22:09:45.108242 3658 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "20c3d6dc17494bcab85cfa79c470a850"
server_key_iv: "7b29366673cc34b4b147502cca5d7b60"
server_key_version: "encryptionkey@0"
I20260811 22:09:45.112679 3658 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:09:45.115579 3675 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:45.117204 3658 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:45.117360 3658 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "20c3d6dc17494bcab85cfa79c470a850"
server_key_iv: "7b29366673cc34b4b147502cca5d7b60"
server_key_version: "encryptionkey@0"
I20260811 22:09:45.117472 3658 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:45.130014 3658 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:45.134038 3658 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:45.134229 3658 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:45.143123 3658 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:42635
I20260811 22:09:45.143518 3741 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:42635 every 8 connection(s)
I20260811 22:09:45.144268 3658 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:45.147462 3743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:45.153885 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 3658
I20260811 22:09:45.153882 3743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f: Bootstrap starting.
I20260811 22:09:45.154036 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:45.154362 28862 external_mini_cluster.cc:1428] Setting key 0ae9fcf63d6361e09276d053ee5a827a
I20260811 22:09:45.156883 3743 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:45.157757 3743 log.cc:824] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:45.160622 3743 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f: No bootstrap required, opened a new log
Aug 11 22:09:45 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:45.166131 3743 raft_consensus.cc:348] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } }
I20260811 22:09:45.166505 3743 raft_consensus.cc:374] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:45.166625 3743 raft_consensus.cc:729] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bfa9d50481124d2e9f97ffe5a3ddc26f, State: Initialized, Role: FOLLOWER
I20260811 22:09:45.167287 3743 consensus_queue.cc:260] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } }
I20260811 22:09:45.167533 3743 raft_consensus.cc:388] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:45.167636 3743 raft_consensus.cc:482] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:45.167810 3743 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:45.169131 3743 raft_consensus.cc:504] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } }
I20260811 22:09:45.169637 3743 leader_election.cc:302] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bfa9d50481124d2e9f97ffe5a3ddc26f; no voters:
I20260811 22:09:45.170080 3743 leader_election.cc:288] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:45.170202 3748 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:45.170436 3748 raft_consensus.cc:686] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [term 1 LEADER]: Becoming Leader. State: Replica: bfa9d50481124d2e9f97ffe5a3ddc26f, State: Running, Role: LEADER
I20260811 22:09:45.170809 3748 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } }
I20260811 22:09:45.171663 3743 sys_catalog.cc:564] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:45.172240 3748 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfa9d50481124d2e9f97ffe5a3ddc26f. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } }
I20260811 22:09:45.172353 3748 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:45.172704 3749 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } }
I20260811 22:09:45.172825 3749 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:45.173413 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:45.155896 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41600 (local address 127.28.47.190:42635)
0811 22:09:45.156407 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:45.156424 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:45.156464 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:09:45.157322 (+ 858us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:45.157344 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:45.157723 (+ 379us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:45.158094 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:45.159105 (+ 1011us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.160536 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.161816 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.162479 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.162614 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:45.166318 (+ 3704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:45.166332 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:45.166349 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:45.166395 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:45.168937 (+ 2542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:45.169546 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:45.169552 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:45.169559 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:45.169698 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:45.170031 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:45.170034 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:45.170036 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:45.170677 (+ 641us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:45.171658 (+ 981us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:45.171818 (+ 160us) server_negotiation.cc:299] Negotiation successful
0811 22:09:45.172170 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":170,"threads_started":1}
I20260811 22:09:45.173468 3753 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:45.177672 3753 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:45.184612 3753 catalog_manager.cc:1347] Generated new cluster ID: 700cadfaed6c4528aaef3158f64f6b98
I20260811 22:09:45.184712 3753 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:45.201129 3753 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:45.202343 3753 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:45.211427 3753 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f: Generated new TSK 0
I20260811 22:09:45.212457 3753 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab.
Entry for principal oryx/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab.
I20260811 22:09:45.258371 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42635
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab
--principal=oryx/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:45.392581 3773 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:45.392912 3773 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:45.392998 3773 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:45.397768 3773 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:45.397855 3773 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:45.397927 3773 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:45.403931 3773 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab
--principal=oryx/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3773
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:45.405293 3773 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:45.406493 3773 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:45.417649 3780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:45.417667 3779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:45.417670 3782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:45.417980 3773 server_base.cc:1034] running on GCE node
I20260811 22:09:45.418480 3773 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:45.419231 3773 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:45.420442 3773 hybrid_clock.cc:648] HybridClock initialized: now 1786486185420398 us; error 56 us; skew 500 ppm
Aug 11 22:09:45 dist-test-slave-vxj2 krb5kdc[3643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486185, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:45.424201 3773 init.cc:377] Logged in from keytab as oryx/127.28.47.129@KRBTEST.COM (short username oryx)
I20260811 22:09:45.425536 3773 webserver.cc:460] Webserver started at http://127.28.47.129:45485/ using document root <none> and password file <none>
I20260811 22:09:45.426213 3773 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:45.426288 3773 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:45.426493 3773 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:45.428773 3773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "0f37525d62de4040b227093cc227011a"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "b819cd6b219d86c9b915dfd87af5347a"
server_key_iv: "d3d5aeb87330faa51c73f88edf2f7d22"
server_key_version: "encryptionkey@0"
I20260811 22:09:45.429462 3773 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "0f37525d62de4040b227093cc227011a"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "b819cd6b219d86c9b915dfd87af5347a"
server_key_iv: "d3d5aeb87330faa51c73f88edf2f7d22"
server_key_version: "encryptionkey@0"
I20260811 22:09:45.433935 3773 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:09:45.436833 3790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:45.438627 3773 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:45.438830 3773 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "0f37525d62de4040b227093cc227011a"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "b819cd6b219d86c9b915dfd87af5347a"
server_key_iv: "d3d5aeb87330faa51c73f88edf2f7d22"
server_key_version: "encryptionkey@0"
I20260811 22:09:45.438987 3773 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:45.454912 3773 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:45.458827 3773 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:45.459086 3773 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:45.459771 3773 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:45.460759 3773 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:45.460845 3773 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:45.460925 3773 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:45.460978 3773 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:45.471280 3773 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:39521
I20260811 22:09:45.471377 3919 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:39521 every 8 connection(s)
I20260811 22:09:45.472437 3773 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:45.477501 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 3773
I20260811 22:09:45.477667 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:09:45.477958 28862 external_mini_cluster.cc:1428] Setting key 9233e7410bb7ace3933ff5f250df1e50
Aug 11 22:09:45 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486185, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:45.492187 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:45.474761 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:40707 (local address 127.28.47.190:42635)
0811 22:09:45.474951 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:45.474956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:45.475860 (+ 904us) server_negotiation.cc:407] Connection header received
0811 22:09:45.477428 (+ 1568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:45.477432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:45.477503 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:45.477614 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:45.479496 (+ 1882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.480315 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.482298 (+ 1983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.482725 (+ 427us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.482832 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:45.487217 (+ 4385us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:45.487223 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:45.487229 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:45.487286 (+ 57us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:45.489332 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:45.490289 (+ 957us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:45.490296 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:45.490301 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:45.490427 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:45.490856 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:45.490860 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:45.490861 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:45.491092 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:45.491214 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:45.491857 (+ 643us) server_negotiation.cc:299] Negotiation successful
0811 22:09:45.492012 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
WARNING: no policy specified for oryx/127.28.47.130@KRBTEST.COM; defaulting to no policy
I20260811 22:09:45.493523 3923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:45.475116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.129:40707)
0811 22:09:45.475652 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:45.475716 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:45.477070 (+ 1354us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:45.477910 (+ 840us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:45.477918 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:45.478496 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:45.479315 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:45.479338 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.480511 (+ 1173us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.480517 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:45.482112 (+ 1595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:45.482129 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.483120 (+ 991us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.483125 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:45.483419 (+ 294us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:45.484300 (+ 881us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:45.484325 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:45.486910 (+ 2585us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:45.489517 (+ 2607us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:45.489528 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:45.489558 (+ 30us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:45.490112 (+ 554us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:45.490567 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:45.490570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:45.490571 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:45.490727 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:45.491215 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:45.491224 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:45.491552 (+ 328us) client_negotiation.cc:769] Sending connection context
0811 22:09:45.491771 (+ 219us) client_negotiation.cc:240] Negotiation successful
0811 22:09:45.492182 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":133,"threads_started":1}
I20260811 22:09:45.495422 3920 heartbeater.cc:347] Connected to a master server at 127.28.47.190:42635
I20260811 22:09:45.495741 3920 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.47.130@KRBTEST.COM" created.
I20260811 22:09:45.496573 3920 heartbeater.cc:511] Master 127.28.47.190:42635 requested a full tablet report, sending...
I20260811 22:09:45.502828 3698 ts_manager.cc:194] Registered new tserver with Master: 0f37525d62de4040b227093cc227011a (127.28.47.129:39521)
I20260811 22:09:45.504781 3698 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:40707
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab.
Entry for principal oryx/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab.
I20260811 22:09:45.518432 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42635
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab
--principal=oryx/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:45.649052 3927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:45.649474 3927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:45.649559 3927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:45.653797 3927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:45.653923 3927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:45.654022 3927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:45.659816 3927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab
--principal=oryx/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:45.661095 3927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:45.662222 3927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:45.672618 3933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:45.672619 3934 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:45.672619 3936 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:45.673254 3927 server_base.cc:1034] running on GCE node
I20260811 22:09:45.673763 3927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:45.674823 3927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:45.676052 3927 hybrid_clock.cc:648] HybridClock initialized: now 1786486185676032 us; error 35 us; skew 500 ppm
Aug 11 22:09:45 dist-test-slave-vxj2 krb5kdc[3643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486185, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:45.679366 3927 init.cc:377] Logged in from keytab as oryx/127.28.47.130@KRBTEST.COM (short username oryx)
I20260811 22:09:45.680413 3927 webserver.cc:460] Webserver started at http://127.28.47.130:37893/ using document root <none> and password file <none>
I20260811 22:09:45.681058 3927 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:45.681159 3927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:45.681450 3927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:45.683750 3927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "f99e42e45a6e4a668e580c173dbd1d61"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "032640de377884a139566d34bb28e65d"
server_key_iv: "42745e6385b771f1f987241ed8d13cf3"
server_key_version: "encryptionkey@0"
I20260811 22:09:45.684412 3927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "f99e42e45a6e4a668e580c173dbd1d61"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "032640de377884a139566d34bb28e65d"
server_key_iv: "42745e6385b771f1f987241ed8d13cf3"
server_key_version: "encryptionkey@0"
I20260811 22:09:45.688997 3927 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 22:09:45.691962 3944 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:45.693589 3927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:45.693774 3927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "f99e42e45a6e4a668e580c173dbd1d61"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "032640de377884a139566d34bb28e65d"
server_key_iv: "42745e6385b771f1f987241ed8d13cf3"
server_key_version: "encryptionkey@0"
I20260811 22:09:45.693928 3927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:45.719795 3927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:45.724002 3927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:45.724258 3927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:45.724969 3927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:45.726058 3927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:45.726142 3927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:45.726226 3927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:45.726270 3927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:45.736780 3927 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:37765
I20260811 22:09:45.736807 4074 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:37765 every 8 connection(s)
I20260811 22:09:45.737950 3927 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:45.738500 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 3927
I20260811 22:09:45.738667 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:09:45.739001 28862 external_mini_cluster.cc:1428] Setting key 290c6af41d52ae8b137c471e9102cc77
Aug 11 22:09:45 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486185, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.28.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.47.131@KRBTEST.COM" created.
I20260811 22:09:45.756413 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:45.740144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:57095 (local address 127.28.47.190:42635)
0811 22:09:45.740293 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:45.740297 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:45.741541 (+ 1244us) server_negotiation.cc:407] Connection header received
0811 22:09:45.743244 (+ 1703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:45.743256 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:45.743312 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:45.743419 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:45.745373 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.745969 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.748154 (+ 2185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.748426 (+ 272us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.748535 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:45.751973 (+ 3438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:45.751976 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:45.751979 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:45.752010 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:45.753596 (+ 1586us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:45.754277 (+ 681us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:45.754280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:45.754281 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:45.754394 (+ 113us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:45.754970 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:45.754973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:45.754974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:45.755263 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:45.755376 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:45.755989 (+ 613us) server_negotiation.cc:299] Negotiation successful
0811 22:09:45.756159 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:09:45.757308 4077 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:45.740688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.130:57095)
0811 22:09:45.741388 (+ 700us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:45.741437 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:45.742872 (+ 1435us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:45.743650 (+ 778us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:45.743662 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:45.744289 (+ 627us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:45.745135 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:45.745191 (+ 56us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.746148 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.746156 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:45.747961 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:45.747981 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.748632 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.748637 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:45.748901 (+ 264us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:45.749688 (+ 787us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:45.749716 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:45.751687 (+ 1971us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:45.753797 (+ 2110us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:45.753804 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:45.753821 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:45.754137 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:45.754559 (+ 422us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:45.754566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:45.754568 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:45.754808 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:45.755380 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:45.755386 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:45.755673 (+ 287us) client_negotiation.cc:769] Sending connection context
0811 22:09:45.755869 (+ 196us) client_negotiation.cc:240] Negotiation successful
0811 22:09:45.756183 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":177,"threads_started":1}
I20260811 22:09:45.758742 4075 heartbeater.cc:347] Connected to a master server at 127.28.47.190:42635
I20260811 22:09:45.759045 4075 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:45.759712 4075 heartbeater.cc:511] Master 127.28.47.190:42635 requested a full tablet report, sending...
I20260811 22:09:45.761226 3698 ts_manager.cc:194] Registered new tserver with Master: f99e42e45a6e4a668e580c173dbd1d61 (127.28.47.130:37765)
I20260811 22:09:45.761909 3698 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:57095
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab.
Entry for principal oryx/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab.
I20260811 22:09:45.776885 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42635
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab
--principal=oryx/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:45.900161 4081 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:45.900516 4081 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:45.900593 4081 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:45.904898 4081 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:45.905027 4081 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:45.905128 4081 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:45.910915 4081 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab
--principal=oryx/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42635
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4081
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:45.912165 4081 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:45.913303 4081 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:45.923904 4091 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:45.923974 4089 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:45.923905 4088 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:45.924221 4081 server_base.cc:1034] running on GCE node
I20260811 22:09:45.924779 4081 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:45.925617 4081 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:45.926887 4081 hybrid_clock.cc:648] HybridClock initialized: now 1786486185926871 us; error 27 us; skew 500 ppm
Aug 11 22:09:45 dist-test-slave-vxj2 krb5kdc[3643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486185, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:45.930187 4081 init.cc:377] Logged in from keytab as oryx/127.28.47.131@KRBTEST.COM (short username oryx)
I20260811 22:09:45.931195 4081 webserver.cc:460] Webserver started at http://127.28.47.131:45567/ using document root <none> and password file <none>
I20260811 22:09:45.931798 4081 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:45.931865 4081 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:45.932061 4081 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:45.934405 4081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "87d7918cb98a4bdf8fb75f538a3ad408"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "2f8bc81a73e93705d7510c8014463493"
server_key_iv: "d019f49c81238aaf85b698ba6371f141"
server_key_version: "encryptionkey@0"
I20260811 22:09:45.934980 4081 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "87d7918cb98a4bdf8fb75f538a3ad408"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "2f8bc81a73e93705d7510c8014463493"
server_key_iv: "d019f49c81238aaf85b698ba6371f141"
server_key_version: "encryptionkey@0"
I20260811 22:09:45.939237 4081 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:09:45.942202 4101 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:45.943953 4081 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:45.944114 4081 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "87d7918cb98a4bdf8fb75f538a3ad408"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "2f8bc81a73e93705d7510c8014463493"
server_key_iv: "d019f49c81238aaf85b698ba6371f141"
server_key_version: "encryptionkey@0"
I20260811 22:09:45.944262 4081 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:45.963938 4081 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:45.967908 4081 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:45.968292 4081 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:45.969250 4081 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:45.970363 4081 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:45.970430 4081 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:45.970521 4081 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:45.970570 4081 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:45.981634 4081 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:39211
I20260811 22:09:45.981699 4236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:39211 every 8 connection(s)
I20260811 22:09:45.982769 4081 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:45.985709 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 4081
I20260811 22:09:45.985860 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:09:45.986158 28862 external_mini_cluster.cc:1428] Setting key 05a1e23059c31d2ffd7b26aa3e6c1eb9
Aug 11 22:09:45 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486185, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:46.000329 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:45.985505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:52329 (local address 127.28.47.190:42635)
0811 22:09:45.985902 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:45.985912 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:45.987030 (+ 1118us) server_negotiation.cc:407] Connection header received
0811 22:09:45.988758 (+ 1728us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:45.988761 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:45.988821 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:45.988901 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:45.990330 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.990902 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.992153 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.992399 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.992520 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:45.995545 (+ 3025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:45.995548 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:45.995551 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:45.995581 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:45.997697 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:45.998425 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:45.998429 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:45.998432 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:45.998533 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:45.999001 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:45.999004 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:45.999005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:45.999234 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:45.999358 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:45.999981 (+ 623us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.000139 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 22:09:46.001533 4240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:45.986091 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.131:52329)
0811 22:09:45.986827 (+ 736us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:45.986921 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:45.988518 (+ 1597us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:45.989103 (+ 585us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:45.989111 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:45.989610 (+ 499us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:45.990172 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:45.990188 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.991049 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.991053 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:45.992012 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:45.992023 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:45.992597 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:45.992602 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:45.992802 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:45.993533 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:45.993553 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:45.995301 (+ 1748us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:45.997898 (+ 2597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:45.997905 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:45.997925 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:45.998300 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:45.998647 (+ 347us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:45.998650 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:45.998653 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:45.998852 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:45.999357 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:45.999367 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:45.999702 (+ 335us) client_negotiation.cc:769] Sending connection context
0811 22:09:45.999935 (+ 233us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.000363 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":445,"thread_start_us":214,"threads_started":1}
I20260811 22:09:46.002910 4238 heartbeater.cc:347] Connected to a master server at 127.28.47.190:42635
I20260811 22:09:46.003192 4238 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:46.003832 4238 heartbeater.cc:511] Master 127.28.47.190:42635 requested a full tablet report, sending...
I20260811 22:09:46.005100 3698 ts_manager.cc:194] Registered new tserver with Master: 87d7918cb98a4bdf8fb75f538a3ad408 (127.28.47.131:39211)
I20260811 22:09:46.005841 3698 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:52329
I20260811 22:09:46.013617 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.28.47.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.47.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.47.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.189.keytab.
Entry for principal oryx/127.28.47.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.189.keytab.
I20260811 22:09:46.049625 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.189:32861
--webserver_interface=127.28.47.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.189.keytab
--principal=oryx/127.28.47.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:42635,127.28.47.189:32861
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:46.176479 4245 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:46.176805 4245 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:46.176849 4245 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:46.181421 4245 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:46.181514 4245 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:46.181533 4245 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:46.181551 4245 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:46.181567 4245 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:46.187649 4245 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:42635,127.28.47.189:32861
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.189.keytab
--principal=oryx/127.28.47.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.189:32861
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.47.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4245
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
W20260811 22:09:46.187952 4245 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.28.47.190:42635,127.28.47.189:32861'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260811 22:09:46.188944 4245 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:46.190116 4245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:46.199687 4255 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:46.199718 4245 server_base.cc:1034] running on GCE node
W20260811 22:09:46.199674 4254 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:46.199667 4258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:46.200619 4245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:46.201946 4245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:46.203142 4245 hybrid_clock.cc:648] HybridClock initialized: now 1786486186203104 us; error 54 us; skew 500 ppm
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486186, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:46.206562 4245 init.cc:377] Logged in from keytab as oryx/127.28.47.189@KRBTEST.COM (short username oryx)
I20260811 22:09:46.207679 4245 webserver.cc:460] Webserver started at http://127.28.47.189:32979/ using document root <none> and password file <none>
I20260811 22:09:46.208338 4245 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:46.208438 4245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:46.208689 4245 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:46.211311 4245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/data/instance:
uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60"
format_stamp: "Formatted at 2026-08-11 22:09:46 on dist-test-slave-vxj2"
server_key: "ed366cc8a6d801662bcd0cff1b7fbf88"
server_key_iv: "7eac44ee293476caaf320c3512badb1b"
server_key_version: "encryptionkey@0"
I20260811 22:09:46.212114 4245 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/wal/instance:
uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60"
format_stamp: "Formatted at 2026-08-11 22:09:46 on dist-test-slave-vxj2"
server_key: "ed366cc8a6d801662bcd0cff1b7fbf88"
server_key_iv: "7eac44ee293476caaf320c3512badb1b"
server_key_version: "encryptionkey@0"
I20260811 22:09:46.216899 4245 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.006s
I20260811 22:09:46.219913 4267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:46.221673 4245 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:46.221838 4245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/wal
uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60"
format_stamp: "Formatted at 2026-08-11 22:09:46 on dist-test-slave-vxj2"
server_key: "ed366cc8a6d801662bcd0cff1b7fbf88"
server_key_iv: "7eac44ee293476caaf320c3512badb1b"
server_key_version: "encryptionkey@0"
I20260811 22:09:46.221971 4245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:46.263077 4245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:46.266999 4245 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:46.267252 4245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:46.276021 4245 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.189:32861
I20260811 22:09:46.276376 4331 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.189:32861 every 8 connection(s)
I20260811 22:09:46.277351 4245 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/data/info.pb
I20260811 22:09:46.279326 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 4245
I20260811 22:09:46.281298 4332 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:46.286649 4332 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.47.189@KRBTEST.COM
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486186, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.189@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:46.297312 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.281131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58138 (local address 127.28.47.189:32861)
0811 22:09:46.281718 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.281735 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.281778 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:09:46.282474 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.282492 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.282886 (+ 394us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.283214 (+ 328us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.284361 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.285574 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.286904 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.287579 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.287722 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.291463 (+ 3741us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.291471 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.291570 (+ 99us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.291611 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.293746 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.294300 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.294307 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.294320 (+ 13us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.294499 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.294863 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.294867 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.294869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.295416 (+ 547us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.295640 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.295958 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.296392 (+ 434us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":185,"threads_started":1}
I20260811 22:09:46.299243 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.289387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41608 (local address 127.28.47.190:42635)
0811 22:09:46.289582 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.289590 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.290029 (+ 439us) server_negotiation.cc:407] Connection header received
0811 22:09:46.290216 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.290219 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.290277 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.290365 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.291089 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.291673 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.292660 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.292906 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.293023 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.295555 (+ 2532us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.295559 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.295562 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.295597 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.297217 (+ 1620us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.297755 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.297766 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.297768 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.297852 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.298139 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.298143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.298144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.298461 (+ 317us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.298576 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.298926 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.299064 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:09:46.299310 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.289351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:41608)
0811 22:09:46.289896 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.289943 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.290104 (+ 161us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.290424 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.290430 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.290807 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.290956 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.290962 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.291840 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.291843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.292530 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.292540 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.293036 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.293040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.293232 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.293757 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.293771 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.295401 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.297344 (+ 1943us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.297352 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.297358 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.297652 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.297951 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.297955 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.297957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.298045 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.298573 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.298581 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.298714 (+ 133us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.298866 (+ 152us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.299064 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"spinlock_wait_cycles":2432,"thread_start_us":235,"threads_started":1}
I20260811 22:09:46.300832 4332 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486186, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.189@KRBTEST.COM for oryx/127.28.47.189@KRBTEST.COM
I20260811 22:09:46.310065 4337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.301497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:32861 (local address 127.0.0.1:58146)
0811 22:09:46.301879 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.301905 (+ 26us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.302027 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.302259 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.302262 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.302454 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.302551 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.302557 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.303758 (+ 1201us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.303761 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.304292 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.304300 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.304740 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.304744 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.304830 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.305448 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.305459 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.306782 (+ 1323us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.308471 (+ 1689us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.308476 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.308479 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.308669 (+ 190us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.308934 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.308939 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.308941 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.309018 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.309485 (+ 467us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.309490 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.309581 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.309683 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.309851 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":178}
I20260811 22:09:46.310063 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.301721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58146 (local address 127.28.47.189:32861)
0811 22:09:46.301851 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.301855 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.301970 (+ 115us) server_negotiation.cc:407] Connection header received
0811 22:09:46.302115 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.302118 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.302162 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.302245 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.302662 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.303631 (+ 969us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.304409 (+ 778us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.304634 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.304741 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.306949 (+ 2208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.306952 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.306955 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.306985 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.308357 (+ 1372us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.308763 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.308768 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.308770 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.308846 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.309104 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.309110 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.309111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.309370 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.309476 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.309741 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.309903 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":60,"server-negotiator.queue_time_us":49}
I20260811 22:09:46.313709 4332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: Bootstrap starting.
I20260811 22:09:46.316416 4332 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:46.317502 4332 log.cc:824] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:46.320526 4332 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: No bootstrap required, opened a new log
I20260811 22:09:46.324045 4332 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } }
I20260811 22:09:46.324285 4332 raft_consensus.cc:374] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:46.324361 4332 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83b6f0f1668c4e0b8e60b5d3d4ea8a60, State: Initialized, Role: FOLLOWER
I20260811 22:09:46.324844 4332 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } }
I20260811 22:09:46.325495 4340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } } }
I20260811 22:09:46.325649 4340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:09:46.326283 4332 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:09:46.331780 4353 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:09:46.331897 4353 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:09:46.334601 4245 master_runner.cc:428] Detected that this master 83b6f0f1668c4e0b8e60b5d3d4ea8a60 is joining an existing cluster
I20260811 22:09:46.334692 4245 master_runner.cc:432] Initiating AddMaster RPC to add 127.28.47.189:32861
I20260811 22:09:46.348073 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.337595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41624 (local address 127.28.47.190:42635)
0811 22:09:46.337828 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.337833 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.338045 (+ 212us) server_negotiation.cc:407] Connection header received
0811 22:09:46.338268 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.338272 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.338324 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.338390 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.339366 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.340231 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.341474 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.341847 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.341965 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.343720 (+ 1755us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.343725 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.343728 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.343769 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.345845 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.346524 (+ 679us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.346528 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.346530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.346627 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.346947 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.346951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.346954 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.347263 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.347477 (+ 214us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.347693 (+ 216us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.347865 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260811 22:09:46.348131 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.337932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58156 (local address 127.28.47.189:32861)
0811 22:09:46.338059 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.338064 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.338163 (+ 99us) server_negotiation.cc:407] Connection header received
0811 22:09:46.338253 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.338256 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.338305 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.338402 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.340396 (+ 1994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.341246 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.342341 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.342666 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.342780 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.344286 (+ 1506us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.344290 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.344293 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.344314 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.346066 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.346639 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.346645 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.346647 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.346746 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.347067 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.347070 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.347072 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.347291 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.347395 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.347841 (+ 446us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.347956 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":50}
I20260811 22:09:46.347995 4359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.337459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:41624)
0811 22:09:46.337956 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.337971 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.338130 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.338547 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.338550 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.338865 (+ 315us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.339221 (+ 356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.339235 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.340392 (+ 1157us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.340395 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.341226 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.341249 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.341997 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.342001 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.342114 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.342951 (+ 837us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.342974 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.343546 (+ 572us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.345990 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.345998 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.346001 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.346392 (+ 391us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.346719 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.346722 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.346724 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.346847 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.347384 (+ 537us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.347387 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.347493 (+ 106us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.347605 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.347779 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":98,"spinlock_wait_cycles":2688,"thread_start_us":104,"threads_started":1}
I20260811 22:09:46.348901 4360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.337459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:32861 (local address 127.0.0.1:58156)
0811 22:09:46.338078 (+ 619us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.338091 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.338175 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.338434 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.338437 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.338860 (+ 423us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.339217 (+ 357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.339226 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.341386 (+ 2160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.341393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.342184 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.342196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.342991 (+ 795us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.342995 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.343102 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.343623 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.343637 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.344155 (+ 518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.346181 (+ 2026us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.346184 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.346187 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.346539 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.346870 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.346873 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.346876 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.346969 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.347441 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.347444 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.347609 (+ 165us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.348025 (+ 416us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.348629 (+ 604us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"mutex_wait_us":153,"spinlock_wait_cycles":40064,"thread_start_us":126,"threads_started":1}
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486185, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.190@KRBTEST.COM for oryx/127.28.47.189@KRBTEST.COM
WARNING: no policy specified for oryx/127.28.47.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.47.188@KRBTEST.COM" created.
I20260811 22:09:46.368049 4362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.354580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:32861 (local address 127.0.0.1:58172)
0811 22:09:46.355047 (+ 467us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.355108 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.355263 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.355952 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.355963 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.356593 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.356844 (+ 251us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.356860 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.357997 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.358004 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.359360 (+ 1356us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.359381 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.360083 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.360088 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.360262 (+ 174us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.361059 (+ 797us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.361084 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.363257 (+ 2173us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.365616 (+ 2359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.365630 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.365639 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.366078 (+ 439us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.366621 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.366624 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.366626 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.366742 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.367207 (+ 465us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.367215 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.367375 (+ 160us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.367583 (+ 208us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.367842 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":151,"threads_started":1}
I20260811 22:09:46.368073 4335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.355456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58172 (local address 127.28.47.189:32861)
0811 22:09:46.355609 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.355614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.355629 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:09:46.355700 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.355703 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.355768 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.355852 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.357011 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.357846 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.359554 (+ 1708us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.359922 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.360031 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.363477 (+ 3446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.363486 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.363493 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.363540 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.365478 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.366211 (+ 733us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.366214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.366215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.366480 (+ 265us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.366846 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.366848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.366850 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.367092 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.367174 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.367662 (+ 488us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.367867 (+ 205us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:09:46.370425 3698 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.28.47.189:32861: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60"
member_type: NON_VOTER
last_known_addr {
host: "127.28.47.189"
port: 32861
}
attrs {
promote: true
}
}
dest_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f"
cas_config_opid_index: -1
I20260811 22:09:46.371454 3698 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true } }
I20260811 22:09:46.374554 4366 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 83b6f0f1668c4e0b8e60b5d3d4ea8a60 (127.28.47.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true } } }
I20260811 22:09:46.375504 4299 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:46.376811 4366 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.28.47.189:32861
I20260811 22:09:46.377534 4245 master.cc:537] Master@127.28.47.189:32861 shutting down...
I20260811 22:09:46.377954 4299 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 FOLLOWER]: Refusing update from remote peer bfa9d50481124d2e9f97ffe5a3ddc26f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260811 22:09:46.378950 4365 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true } } }
I20260811 22:09:46.379092 4365 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:46.379936 4365 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:09:46.384011 3678 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
I20260811 22:09:46.384653 4245 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:46.385004 4245 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:46.385144 4245 tablet_replica.cc:330] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: stopping tablet replica
W20260811 22:09:46.385689 3678 consensus_peers.cc:487] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f -> Peer 83b6f0f1668c4e0b8e60b5d3d4ea8a60 (127.28.47.189:32861): Couldn't send request to peer 83b6f0f1668c4e0b8e60b5d3d4ea8a60. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.47.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.188.keytab.
Entry for principal oryx/127.28.47.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.188.keytab.
I20260811 22:09:46.389647 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.188:35257
--webserver_interface=127.28.47.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.188.keytab
--principal=oryx/127.28.47.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:42635,127.28.47.189:32861,127.28.47.188:35257
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:09:46.401237 4245 master.cc:552] Master@127.28.47.189:32861 shutdown complete.
I20260811 22:09:46.401325 4245 master_runner.cc:305] Clearing existing system tablet
I20260811 22:09:46.401644 4245 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:46.401719 4245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:46.403254 4245 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 22:09:46.404228 4372 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:46.404481 4245 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:46.404613 4245 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:09:46.404647 4245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/wal
uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60"
format_stamp: "Formatted at 2026-08-11 22:09:46 on dist-test-slave-vxj2"
server_key: "ed366cc8a6d801662bcd0cff1b7fbf88"
server_key_iv: "7eac44ee293476caaf320c3512badb1b"
server_key_version: "encryptionkey@0"
I20260811 22:09:46.407135 4245 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:46.409981 4245 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:09:46.410095 4245 log.cc:1196] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260811 22:09:46.410365 4245 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: Deleting consensus metadata
I20260811 22:09:46.410547 4245 master_runner.cc:315] Copying system tablet from 127.28.47.190:42635
I20260811 22:09:46.412046 4245 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: tablet copy: Beginning tablet copy session from remote peer at address 127.28.47.190:42635
I20260811 22:09:46.421928 4378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.412750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:41626)
0811 22:09:46.413093 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.413108 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.413278 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.413596 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.413599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.413877 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.414019 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.414027 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.415147 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.415151 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.416081 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.416092 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.416726 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.416729 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.416836 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.417439 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.417455 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.417899 (+ 444us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.420026 (+ 2127us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.420030 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.420032 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.420309 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.420626 (+ 317us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.420629 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.420631 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.420724 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.421319 (+ 595us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.421322 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.421439 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.421546 (+ 107us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.421696 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":97,"threads_started":1}
I20260811 22:09:46.422048 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.412888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41626 (local address 127.28.47.190:42635)
0811 22:09:46.413030 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.413035 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.413216 (+ 181us) server_negotiation.cc:407] Connection header received
0811 22:09:46.413395 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.413399 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.413454 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.413557 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.414161 (+ 604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.415021 (+ 860us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.416257 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.416588 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.416722 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.418060 (+ 1338us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.418064 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.418067 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.418096 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.419903 (+ 1807us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.420425 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.420428 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.420430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.420527 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.420819 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.420822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.420824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.421216 (+ 392us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.421323 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.421626 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.421752 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:09:46.423118 3722 tablet_copy_service.cc:140] P bfa9d50481124d2e9f97ffe5a3ddc26f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 83b6f0f1668c4e0b8e60b5d3d4ea8a60 ({username='oryx', principal='oryx/127.28.47.189@KRBTEST.COM'} at 127.0.0.1:41626)
I20260811 22:09:46.423319 3722 tablet_copy_service.cc:161] P bfa9d50481124d2e9f97ffe5a3ddc26f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 83b6f0f1668c4e0b8e60b5d3d4ea8a60 at {username='oryx', principal='oryx/127.28.47.189@KRBTEST.COM'} at 127.0.0.1:41626: session id = 83b6f0f1668c4e0b8e60b5d3d4ea8a60-00000000000000000000000000000000
I20260811 22:09:46.425331 3722 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:09:46.427491 4245 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:09:46.430390 4245 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: tablet copy: Starting download of 0 data blocks...
I20260811 22:09:46.430724 4245 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: tablet copy: Starting download of 1 WAL segments...
I20260811 22:09:46.433001 4245 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:09:46.435415 3722 tablet_copy_service.cc:342] P bfa9d50481124d2e9f97ffe5a3ddc26f: Request end of tablet copy session 83b6f0f1668c4e0b8e60b5d3d4ea8a60-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.28.47.189@KRBTEST.COM'} at 127.0.0.1:41626
I20260811 22:09:46.435627 3722 tablet_copy_service.cc:433] P bfa9d50481124d2e9f97ffe5a3ddc26f: ending tablet copy session 83b6f0f1668c4e0b8e60b5d3d4ea8a60-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 83b6f0f1668c4e0b8e60b5d3d4ea8a60
I20260811 22:09:46.444517 4245 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:46.446712 4379 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:46.450512 4380 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:46.450695 4245 server_base.cc:1034] running on GCE node
W20260811 22:09:46.453364 4382 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:46.453676 4245 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:46.454173 4245 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:46.455327 4245 hybrid_clock.cc:648] HybridClock initialized: now 1786486186455305 us; error 43 us; skew 500 ppm
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486186, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:46.457051 4245 init.cc:377] Logged in from keytab as oryx/127.28.47.189@KRBTEST.COM (short username oryx)
I20260811 22:09:46.457650 4245 webserver.cc:460] Webserver started at http://127.28.47.189:45389/ using document root <none> and password file <none>
I20260811 22:09:46.457857 4245 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:46.457908 4245 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:46.459488 4245 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260811 22:09:46.460647 4391 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:46.460954 4245 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 22:09:46.461035 4245 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/wal
uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60"
format_stamp: "Formatted at 2026-08-11 22:09:46 on dist-test-slave-vxj2"
server_key: "ed366cc8a6d801662bcd0cff1b7fbf88"
server_key_iv: "7eac44ee293476caaf320c3512badb1b"
server_key_version: "encryptionkey@0"
I20260811 22:09:46.461122 4245 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:46.476169 4245 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:46.478747 4245 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:46.484639 4245 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.189:32861
I20260811 22:09:46.484673 4463 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.189:32861 every 8 connection(s)
I20260811 22:09:46.485370 4245 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-1/data/info.pb
I20260811 22:09:46.487409 4464 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:09:46.488554 4464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: Bootstrap starting.
I20260811 22:09:46.500533 4464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:09:46.500901 4464 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: Bootstrap complete.
I20260811 22:09:46.501595 4464 raft_consensus.cc:348] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true } }
I20260811 22:09:46.501745 4464 raft_consensus.cc:729] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 83b6f0f1668c4e0b8e60b5d3d4ea8a60, State: Initialized, Role: LEARNER
I20260811 22:09:46.501858 4464 consensus_queue.cc:260] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true } }
I20260811 22:09:46.502377 4464 sys_catalog.cc:564] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:46.502934 4468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true } } }
I20260811 22:09:46.503180 4468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:09:46.505693 4482 catalog_manager.cc:1259] Loaded cluster ID: 700cadfaed6c4528aaef3158f64f6b98
I20260811 22:09:46.505903 4482 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: loading cluster ID for follower catalog manager: success
I20260811 22:09:46.508117 3920 heartbeater.cc:503] Master 127.28.47.190:42635 was elected leader, sending a full tablet report...
I20260811 22:09:46.508479 4482 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: acquiring CA information for follower catalog manager: success
I20260811 22:09:46.510462 4482 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 22:09:46.541518 4369 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:46.541851 4369 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:46.541895 4369 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:46.546355 4369 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:46.546443 4369 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:46.546484 4369 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:46.546502 4369 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:46.546571 4369 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:46.552666 4369 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:42635,127.28.47.189:32861,127.28.47.188:35257
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.188.keytab
--principal=oryx/127.28.47.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.188:35257
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.47.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4369
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:46.554011 4369 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:46.555176 4369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:46.564634 4490 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:46.564633 4489 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:46.564634 4492 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:46.564838 4369 server_base.cc:1034] running on GCE node
I20260811 22:09:46.565713 4369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:46.567446 4369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:46.568641 4369 hybrid_clock.cc:648] HybridClock initialized: now 1786486186568607 us; error 49 us; skew 500 ppm
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486186, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:46.572196 4369 init.cc:377] Logged in from keytab as oryx/127.28.47.188@KRBTEST.COM (short username oryx)
I20260811 22:09:46.573684 4369 webserver.cc:460] Webserver started at http://127.28.47.188:37019/ using document root <none> and password file <none>
I20260811 22:09:46.574349 4369 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:46.574421 4369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:46.574613 4369 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:46.576951 4369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/data/instance:
uuid: "70a6d0aaab4b4d6b8f6da676b5016064"
format_stamp: "Formatted at 2026-08-11 22:09:46 on dist-test-slave-vxj2"
server_key: "ce79eeaf3c65283e95d7d6c0f7a5f959"
server_key_iv: "c281fd8499d0e6306e496497cfe33924"
server_key_version: "encryptionkey@0"
I20260811 22:09:46.577786 4369 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/wal/instance:
uuid: "70a6d0aaab4b4d6b8f6da676b5016064"
format_stamp: "Formatted at 2026-08-11 22:09:46 on dist-test-slave-vxj2"
server_key: "ce79eeaf3c65283e95d7d6c0f7a5f959"
server_key_iv: "c281fd8499d0e6306e496497cfe33924"
server_key_version: "encryptionkey@0"
I20260811 22:09:46.581967 4369 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:09:46.584657 4499 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:46.586284 4369 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:46.586505 4369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/wal
uuid: "70a6d0aaab4b4d6b8f6da676b5016064"
format_stamp: "Formatted at 2026-08-11 22:09:46 on dist-test-slave-vxj2"
server_key: "ce79eeaf3c65283e95d7d6c0f7a5f959"
server_key_iv: "c281fd8499d0e6306e496497cfe33924"
server_key_version: "encryptionkey@0"
I20260811 22:09:46.586647 4369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:46.604960 4369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:46.608811 4369 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:46.609068 4369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:46.619402 4369 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.188:35257
I20260811 22:09:46.619845 4557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.188:35257 every 8 connection(s)
I20260811 22:09:46.620828 4369 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/data/info.pb
I20260811 22:09:46.624560 4558 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:46.628479 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 4369
I20260811 22:09:46.630169 4558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.47.188@KRBTEST.COM
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486186, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.188@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:46.645656 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.632625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41642 (local address 127.28.47.190:42635)
0811 22:09:46.632781 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.632785 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.633547 (+ 762us) server_negotiation.cc:407] Connection header received
0811 22:09:46.633732 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.633735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.633777 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.633879 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.634906 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.635430 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.637034 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.637429 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.637606 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.641457 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.641460 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.641462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.641500 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.643440 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.644080 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.644084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.644086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.644174 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.644582 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.644584 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.644585 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.644773 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.644860 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.645310 (+ 450us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.645458 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:09:46.646827 4561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.632366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:41642)
0811 22:09:46.633394 (+ 1028us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.633453 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.633613 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.633935 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.633941 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.634431 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.634745 (+ 314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.634758 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.635997 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.636006 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.636842 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.636891 (+ 49us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.637661 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.637666 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.637874 (+ 208us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.638724 (+ 850us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.638748 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.641216 (+ 2468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.643554 (+ 2338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.643560 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.643569 (+ 9us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.643979 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.644283 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.644286 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.644289 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.644481 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.644881 (+ 400us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.644888 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.645046 (+ 158us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.645253 (+ 207us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.645696 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":575,"mutex_wait_us":107,"spinlock_wait_cycles":768,"thread_start_us":303,"threads_started":1}
I20260811 22:09:46.646893 4560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.630231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52364 (local address 127.28.47.188:35257)
0811 22:09:46.630755 (+ 524us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.630776 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.630835 (+ 59us) server_negotiation.cc:407] Connection header received
0811 22:09:46.631782 (+ 947us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.631817 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.632219 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.632652 (+ 433us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.633779 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.635097 (+ 1318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.636074 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.636758 (+ 684us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.636979 (+ 221us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.640334 (+ 3355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.640342 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.640359 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.640390 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.642956 (+ 2566us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.643548 (+ 592us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.643555 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.643563 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.643821 (+ 258us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.644277 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.644280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.644282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.644921 (+ 639us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.645116 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.645396 (+ 280us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.645641 (+ 245us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":354,"spinlock_wait_cycles":2432,"thread_start_us":149,"threads_started":1}
I20260811 22:09:46.648969 4558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486186, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.188@KRBTEST.COM for oryx/127.28.47.189@KRBTEST.COM
I20260811 22:09:46.659504 4561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.649601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:32861 (local address 127.0.0.1:58186)
0811 22:09:46.649748 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.649764 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.649857 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.650404 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.650407 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.650652 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.650924 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.650932 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.652014 (+ 1082us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.652018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.652606 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.652615 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.653261 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.653266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.653456 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.654027 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.654039 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.655540 (+ 1501us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.657807 (+ 2267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.657811 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.657812 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.658058 (+ 246us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.658364 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.658367 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.658369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.658465 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.659032 (+ 567us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.659034 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.659136 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.659232 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.659365 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:09:46.659611 4562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.649765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58186 (local address 127.28.47.189:32861)
0811 22:09:46.650131 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.650135 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.650148 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:09:46.650187 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.650189 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.650230 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.650320 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.651071 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.651855 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.652761 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.653039 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.653135 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.655729 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.655733 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.655735 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.655764 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.657616 (+ 1852us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.658157 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.658160 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.658161 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.658249 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.658594 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.658597 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.658598 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.658917 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.659019 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.659328 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.659464 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":284,"thread_start_us":168,"threads_started":1}
I20260811 22:09:46.660821 4558 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486186, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.188@KRBTEST.COM for oryx/127.28.47.188@KRBTEST.COM
I20260811 22:09:46.669456 4561 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.661468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:35257 (local address 127.0.0.1:52374)
0811 22:09:46.661611 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.661628 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.661725 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.661949 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.661951 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.662103 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.662200 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.662206 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.663262 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.663265 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.663824 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.663833 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.664279 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.664282 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.664342 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.664870 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.664885 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.666352 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.667971 (+ 1619us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.667973 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.667975 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.668171 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.668430 (+ 259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.668432 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.668433 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.668519 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.668939 (+ 420us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.668941 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.669031 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.669123 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.669316 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:09:46.669550 4560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.661469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52374 (local address 127.28.47.188:35257)
0811 22:09:46.661611 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.661616 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.661675 (+ 59us) server_negotiation.cc:407] Connection header received
0811 22:09:46.661797 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.661801 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.661840 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.661919 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.662368 (+ 449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.663117 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.663948 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.664176 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.664261 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.666515 (+ 2254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.666519 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.666520 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.666543 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.667860 (+ 1317us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.668263 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.668266 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.668267 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.668342 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.668607 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.668609 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.668610 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.668842 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.668923 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.669301 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.669407 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:09:46.674337 4558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: Bootstrap starting.
I20260811 22:09:46.677709 4558 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:46.679044 4558 log.cc:824] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:46.681727 4558 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: No bootstrap required, opened a new log
I20260811 22:09:46.686311 4558 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } }
I20260811 22:09:46.686715 4558 raft_consensus.cc:374] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:46.686825 4558 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 70a6d0aaab4b4d6b8f6da676b5016064, State: Initialized, Role: FOLLOWER
I20260811 22:09:46.687352 4558 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } }
I20260811 22:09:46.688201 4565 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } } }
I20260811 22:09:46.688352 4565 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:09:46.688936 4558 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:09:46.696885 4579 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:09:46.697009 4579 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:09:46.700903 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 3773
I20260811 22:09:46.700968 4369 master_runner.cc:428] Detected that this master 70a6d0aaab4b4d6b8f6da676b5016064 is joining an existing cluster
I20260811 22:09:46.701073 4369 master_runner.cc:432] Initiating AddMaster RPC to add 127.28.47.188:35257
I20260811 22:09:46.714437 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:39521
--local_ip_for_outbound_sockets=127.28.47.129
--tserver_master_addrs=127.28.47.190:42635
--webserver_port=45485
--webserver_interface=127.28.47.129
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab
--principal=oryx/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.47.190:42635,127.28.47.189:32861
--tserver_master_addrs=127.28.47.190:42635,127.28.47.189:32861,127.28.47.188:35257 with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:09:46.715791 4560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.706781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52384 (local address 127.28.47.188:35257)
0811 22:09:46.706882 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.706886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.706906 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:09:46.706938 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.706940 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.706974 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.707063 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.707657 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.708155 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.708979 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.709329 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.709441 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.711053 (+ 1612us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.711057 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.711060 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.711082 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.712608 (+ 1526us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.713043 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.713048 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.713050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.713125 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.714245 (+ 1120us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.714248 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.714249 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.714560 (+ 311us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.714648 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.715411 (+ 763us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.715549 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260811 22:09:46.716601 4586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.705574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:35257 (local address 127.0.0.1:52384)
0811 22:09:46.706040 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.706057 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.706207 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.707075 (+ 868us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.707078 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.707385 (+ 307us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.707568 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.707577 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.708260 (+ 683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.708265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.708861 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.708870 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.709662 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.709668 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.709827 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.710536 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.710547 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.710914 (+ 367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.712738 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.712741 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.712744 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.712945 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.714049 (+ 1104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.714051 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.714053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.714152 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.715094 (+ 942us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.715099 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.715223 (+ 124us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.716295 (+ 1072us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.716416 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":372,"thread_start_us":159,"threads_started":1}
I20260811 22:09:46.718943 4565 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:46.719116 4565 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } }
I20260811 22:09:46.720199 4565 leader_election.cc:288] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bfa9d50481124d2e9f97ffe5a3ddc26f (127.28.47.190:42635), 83b6f0f1668c4e0b8e60b5d3d4ea8a60 (127.28.47.189:32861)
I20260811 22:09:46.720849 3709 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" is_pre_election: true
I20260811 22:09:46.721062 4430 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" is_pre_election: true
I20260811 22:09:46.721289 4430 raft_consensus.cc:1799] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 LEARNER]: Handling vote request from an unknown peer 70a6d0aaab4b4d6b8f6da676b5016064
I20260811 22:09:46.721434 4430 raft_consensus.cc:2396] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 70a6d0aaab4b4d6b8f6da676b5016064 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260811 22:09:46.722215 4501 leader_election.cc:302] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 70a6d0aaab4b4d6b8f6da676b5016064; no voters: 83b6f0f1668c4e0b8e60b5d3d4ea8a60, bfa9d50481124d2e9f97ffe5a3ddc26f
I20260811 22:09:46.722708 4565 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:46.724076 4565 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 22:09:46.732170 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.709293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41646 (local address 127.28.47.190:42635)
0811 22:09:46.709599 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.709603 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.709621 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:09:46.709672 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.709675 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.709725 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.709815 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.714082 (+ 4267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.714983 (+ 901us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.716820 (+ 1837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.717346 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.717490 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.719323 (+ 1833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.719327 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.719329 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.719361 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.722456 (+ 3095us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.723097 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.723102 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.723106 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.723208 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.730765 (+ 7557us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.730786 (+ 21us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.730790 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.731147 (+ 357us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.731252 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.731731 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.731916 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219}
I20260811 22:09:46.733228 4585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.704944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:41646)
0811 22:09:46.705370 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.705402 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.705517 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.713254 (+ 7737us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.713257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.713625 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.713913 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.713924 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.715141 (+ 1217us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.715144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.716620 (+ 1476us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.716632 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.717666 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.717672 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.717821 (+ 149us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.718380 (+ 559us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.718401 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.719146 (+ 745us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.722596 (+ 3450us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.722602 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.722607 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.722908 (+ 301us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.723337 (+ 429us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.723342 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.723348 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.730589 (+ 7241us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.731344 (+ 755us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.731350 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.731485 (+ 135us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.732748 (+ 1263us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.732954 (+ 206us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":146,"threads_started":1}
I20260811 22:09:46.739573 4587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.706246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:32861 (local address 127.0.0.1:58198)
0811 22:09:46.707387 (+ 1141us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.707401 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.707516 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.710238 (+ 2722us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.710243 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.729592 (+ 19349us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.729789 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.729801 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.731182 (+ 1381us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.731185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.732152 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.732163 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.732950 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.732953 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.733052 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.733669 (+ 617us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.733684 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.734122 (+ 438us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.736883 (+ 2761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.736887 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.736889 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.738087 (+ 1198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.738430 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.738433 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.738435 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.738525 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.738987 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.738990 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.739086 (+ 96us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.739176 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.739371 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1073,"thread_start_us":185,"threads_started":1}
I20260811 22:09:46.740111 4562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.709456 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58198 (local address 127.28.47.189:32861)
0811 22:09:46.710003 (+ 547us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.710009 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.710024 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:09:46.710066 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.710069 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.710115 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.710200 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.730149 (+ 19949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.731017 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.732348 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.732717 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.732816 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.734295 (+ 1479us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.734299 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.734302 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.734329 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.736377 (+ 2048us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.738210 (+ 1833us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.738214 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.738215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.738326 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.738620 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.738623 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.738625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.738855 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.738945 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.739816 (+ 871us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.739956 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459}
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486185, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.190@KRBTEST.COM for oryx/127.28.47.188@KRBTEST.COM
I20260811 22:09:46.752312 4362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.741519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:35257 (local address 127.0.0.1:52400)
0811 22:09:46.741873 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.741893 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.741994 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.742563 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.742566 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.742782 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.742910 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.742917 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.743985 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.743988 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.744896 (+ 908us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.744909 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.745657 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.745661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.745766 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.746364 (+ 598us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.746380 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.748254 (+ 1874us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.750519 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.750523 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.750526 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.750812 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.751154 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.751157 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.751159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.751259 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.751717 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.751720 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.751839 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.751951 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.752120 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":199}
I20260811 22:09:46.752311 4560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.742182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52400 (local address 127.28.47.188:35257)
0811 22:09:46.742303 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.742307 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.742320 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:09:46.742388 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.742390 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.742434 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.742538 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.743042 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.743851 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.745038 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.745419 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.745519 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.748487 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.748492 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.748496 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.748530 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.750365 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.750939 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.750943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.750945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.751041 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.751362 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.751365 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.751367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.751608 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.751708 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.752025 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.752157 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 22:09:46.753922 3698 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.28.47.188:35257: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064"
member_type: NON_VOTER
last_known_addr {
host: "127.28.47.188"
port: 35257
}
attrs {
promote: true
}
}
dest_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f"
cas_config_opid_index: 5
I20260811 22:09:46.754611 3698 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } }
I20260811 22:09:46.756145 4365 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 70a6d0aaab4b4d6b8f6da676b5016064 (127.28.47.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } } }
I20260811 22:09:46.757853 4365 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.28.47.188:35257
I20260811 22:09:46.758051 4365 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } } }
I20260811 22:09:46.758149 4365 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:46.759037 4369 master.cc:537] Master@127.28.47.188:35257 shutting down...
W20260811 22:09:46.762352 3677 consensus_peers.cc:487] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f -> Peer 70a6d0aaab4b4d6b8f6da676b5016064 (127.28.47.188:35257): Couldn't send request to peer 70a6d0aaab4b4d6b8f6da676b5016064. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 22:09:46.763455 4362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.756384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:32861 (local address 127.0.0.1:58214)
0811 22:09:46.756976 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.756991 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.757084 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.757577 (+ 493us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.757582 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.757603 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:46.757702 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.757708 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.758907 (+ 1199us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.758911 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.760222 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.760237 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.762963 (+ 2726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.762967 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.763077 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.763146 (+ 69us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.763222 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.763306 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512}
I20260811 22:09:46.764544 4562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.756672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58214 (local address 127.28.47.189:32861)
0811 22:09:46.756872 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.756877 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.757192 (+ 315us) server_negotiation.cc:407] Connection header received
0811 22:09:46.757248 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.757251 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.757314 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.757433 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:46.757827 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.758649 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.761020 (+ 2371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.761773 (+ 753us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.761882 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.762063 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.764322 (+ 2259us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.764425 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 22:09:46.765070 4075 heartbeater.cc:503] Master 127.28.47.190:42635 was elected leader, sending a full tablet report...
I20260811 22:09:46.765424 4430 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 LEARNER]: Refusing update from remote peer bfa9d50481124d2e9f97ffe5a3ddc26f: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260811 22:09:46.765962 4366 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260811 22:09:46.768155 4430 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 70a6d0aaab4b4d6b8f6da676b5016064 (127.28.47.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } } }
I20260811 22:09:46.769718 4369 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:46.770036 4369 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:46.770140 4369 tablet_replica.cc:330] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: stopping tablet replica
I20260811 22:09:46.770576 4468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfa9d50481124d2e9f97ffe5a3ddc26f. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } } }
I20260811 22:09:46.770697 4468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:09:46.770792 4468 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: NON_VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: true } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } } }
I20260811 22:09:46.770862 4468 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:09:46.771739 4366 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f: attempting to promote NON_VOTER 83b6f0f1668c4e0b8e60b5d3d4ea8a60 to VOTER
I20260811 22:09:46.772204 4366 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } }
W20260811 22:09:46.774171 3677 consensus_peers.cc:487] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f -> Peer 70a6d0aaab4b4d6b8f6da676b5016064 (127.28.47.188:35257): Couldn't send request to peer 70a6d0aaab4b4d6b8f6da676b5016064. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260811 22:09:46.774665 4431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 LEARNER]: Refusing update from remote peer bfa9d50481124d2e9f97ffe5a3ddc26f: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260811 22:09:46.775300 4590 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260811 22:09:46.777561 4431 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 83b6f0f1668c4e0b8e60b5d3d4ea8a60 (127.28.47.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } } }
I20260811 22:09:46.777109 4593 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 83b6f0f1668c4e0b8e60b5d3d4ea8a60 (127.28.47.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } } }
I20260811 22:09:46.779316 4592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } } }
I20260811 22:09:46.779579 4592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:09:46.781093 4593 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } } }
I20260811 22:09:46.781297 4593 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:46.781383 4365 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } } }
I20260811 22:09:46.781497 4365 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:46.787320 4369 master.cc:552] Master@127.28.47.188:35257 shutdown complete.
I20260811 22:09:46.787401 4369 master_runner.cc:305] Clearing existing system tablet
I20260811 22:09:46.787652 4369 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:46.787714 4369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:46.789180 4369 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 22:09:46.790199 4595 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:46.790506 4369 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:46.790633 4369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260811 22:09:46.790714 4369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/wal
uuid: "70a6d0aaab4b4d6b8f6da676b5016064"
format_stamp: "Formatted at 2026-08-11 22:09:46 on dist-test-slave-vxj2"
server_key: "ce79eeaf3c65283e95d7d6c0f7a5f959"
server_key_iv: "c281fd8499d0e6306e496497cfe33924"
server_key_version: "encryptionkey@0"
I20260811 22:09:46.792811 4369 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:46.795431 4369 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260811 22:09:46.795534 4369 log.cc:1196] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260811 22:09:46.795797 4369 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: Deleting consensus metadata
I20260811 22:09:46.796108 4369 master_runner.cc:315] Copying system tablet from 127.28.47.190:42635
I20260811 22:09:46.797369 4369 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: tablet copy: Beginning tablet copy session from remote peer at address 127.28.47.190:42635
I20260811 22:09:46.807132 4602 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.798044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:41650)
0811 22:09:46.798334 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.798351 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.798471 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.798872 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.798875 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:46.799101 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:46.799233 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.799240 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.800311 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.800315 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.801280 (+ 965us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:46.801291 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.801968 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.801972 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:46.802076 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.802671 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:46.802686 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:46.803144 (+ 458us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:46.805381 (+ 2237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.805385 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.805387 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.805655 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.805979 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:46.805982 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:46.805984 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:46.806078 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:46.806540 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:46.806543 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:46.806655 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 22:09:46.806766 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 22:09:46.806933 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":99,"threads_started":1}
I20260811 22:09:46.807137 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.798465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41650 (local address 127.28.47.190:42635)
0811 22:09:46.798591 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.798596 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.798611 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:09:46.798670 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.798672 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.798743 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.798841 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:46.799368 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.800188 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.801451 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:46.801842 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:46.801957 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:46.803351 (+ 1394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:46.803355 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:46.803358 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:46.803388 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:46.805260 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.805767 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.805770 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.805772 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.805877 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:46.806177 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:46.806181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:46.806182 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:46.806424 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:46.806540 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:46.806851 (+ 311us) server_negotiation.cc:299] Negotiation successful
0811 22:09:46.806981 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:09:46.808424 3722 tablet_copy_service.cc:140] P bfa9d50481124d2e9f97ffe5a3ddc26f: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 70a6d0aaab4b4d6b8f6da676b5016064 ({username='oryx', principal='oryx/127.28.47.188@KRBTEST.COM'} at 127.0.0.1:41650)
I20260811 22:09:46.808545 3722 tablet_copy_service.cc:161] P bfa9d50481124d2e9f97ffe5a3ddc26f: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 70a6d0aaab4b4d6b8f6da676b5016064 at {username='oryx', principal='oryx/127.28.47.188@KRBTEST.COM'} at 127.0.0.1:41650: session id = 70a6d0aaab4b4d6b8f6da676b5016064-00000000000000000000000000000000
I20260811 22:09:46.809788 3722 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f: Tablet Copy: opened 0 blocks and 1 log segments
I20260811 22:09:46.811707 4369 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260811 22:09:46.814553 4369 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: tablet copy: Starting download of 0 data blocks...
I20260811 22:09:46.814781 4369 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: tablet copy: Starting download of 1 WAL segments...
I20260811 22:09:46.816830 4369 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260811 22:09:46.819229 3722 tablet_copy_service.cc:342] P bfa9d50481124d2e9f97ffe5a3ddc26f: Request end of tablet copy session 70a6d0aaab4b4d6b8f6da676b5016064-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.28.47.188@KRBTEST.COM'} at 127.0.0.1:41650
I20260811 22:09:46.819346 3722 tablet_copy_service.cc:433] P bfa9d50481124d2e9f97ffe5a3ddc26f: ending tablet copy session 70a6d0aaab4b4d6b8f6da676b5016064-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 70a6d0aaab4b4d6b8f6da676b5016064
I20260811 22:09:46.829070 4369 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:46.831099 4603 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:46.831336 4604 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:46.831447 4606 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:46.832087 4369 server_base.cc:1034] running on GCE node
I20260811 22:09:46.832273 4369 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:46.832612 4369 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:46.833763 4369 hybrid_clock.cc:648] HybridClock initialized: now 1786486186833749 us; error 32 us; skew 500 ppm
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486186, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:46.835597 4369 init.cc:377] Logged in from keytab as oryx/127.28.47.188@KRBTEST.COM (short username oryx)
I20260811 22:09:46.836133 4369 webserver.cc:460] Webserver started at http://127.28.47.188:38893/ using document root <none> and password file <none>
I20260811 22:09:46.836339 4369 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:46.836387 4369 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:46.838115 4369 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260811 22:09:46.839357 4617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:46.839740 4369 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260811 22:09:46.839828 4369 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/wal
uuid: "70a6d0aaab4b4d6b8f6da676b5016064"
format_stamp: "Formatted at 2026-08-11 22:09:46 on dist-test-slave-vxj2"
server_key: "ce79eeaf3c65283e95d7d6c0f7a5f959"
server_key_iv: "c281fd8499d0e6306e496497cfe33924"
server_key_version: "encryptionkey@0"
I20260811 22:09:46.839923 4369 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:46.848021 4369 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:46.850720 4369 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:46.856330 4369 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.188:35257
I20260811 22:09:46.856361 4682 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.188:35257 every 8 connection(s)
I20260811 22:09:46.857044 4369 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/master-2/data/info.pb
I20260811 22:09:46.859246 4683 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:09:46.860497 4683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: Bootstrap starting.
I20260811 22:09:46.874492 4683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:09:46.875016 4683 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: Bootstrap complete.
I20260811 22:09:46.875825 4683 raft_consensus.cc:348] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } }
I20260811 22:09:46.876273 4683 raft_consensus.cc:729] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 70a6d0aaab4b4d6b8f6da676b5016064, State: Initialized, Role: LEARNER
I20260811 22:09:46.876489 4683 consensus_queue.cc:260] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } }
I20260811 22:09:46.877403 4683 sys_catalog.cc:564] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:46.877357 4686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } } }
I20260811 22:09:46.877652 4686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:09:46.882987 4699 catalog_manager.cc:1259] Loaded cluster ID: 700cadfaed6c4528aaef3158f64f6b98
I20260811 22:09:46.883066 4699 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: loading cluster ID for follower catalog manager: success
I20260811 22:09:46.885754 4699 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: acquiring CA information for follower catalog manager: success
I20260811 22:09:46.888016 4699 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260811 22:09:46.891502 4589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:46.891834 4589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:46.891919 4589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:46.896394 4589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:46.896482 4589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:46.896580 4589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:46.903687 4589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab
--principal=oryx/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:39521
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=45485
--tserver_master_addrs=127.28.47.190:42635,127.28.47.189:32861,127.28.47.188:35257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:46.905062 4589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:46.906332 4589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:46.919248 4706 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:46.919595 4589 server_base.cc:1034] running on GCE node
W20260811 22:09:46.919674 4705 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:46.919807 4708 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:46.920353 4589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:46.921144 4589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:46.922365 4589 hybrid_clock.cc:648] HybridClock initialized: now 1786486186922340 us; error 39 us; skew 500 ppm
Aug 11 22:09:46 dist-test-slave-vxj2 krb5kdc[3643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486186, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:46.926218 4589 init.cc:377] Logged in from keytab as oryx/127.28.47.129@KRBTEST.COM (short username oryx)
I20260811 22:09:46.927359 4589 webserver.cc:460] Webserver started at http://127.28.47.129:45485/ using document root <none> and password file <none>
I20260811 22:09:46.927958 4589 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:46.928035 4589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:46.932641 4589 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:09:46.935388 4716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:46.937124 4589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:46.937355 4589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "0f37525d62de4040b227093cc227011a"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "b819cd6b219d86c9b915dfd87af5347a"
server_key_iv: "d3d5aeb87330faa51c73f88edf2f7d22"
server_key_version: "encryptionkey@0"
I20260811 22:09:46.937987 4589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:46.955790 4589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:46.959900 4589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:46.960168 4589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:46.960896 4589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:46.962034 4589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:46.962114 4589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:46.962190 4589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:46.962239 4589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:46.981248 4589 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:39521
I20260811 22:09:46.981464 4847 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:39521 every 8 connection(s)
I20260811 22:09:46.982520 4589 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:46.987918 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 4589
I20260811 22:09:46.988176 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 3927
I20260811 22:09:47.005462 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:37765
--local_ip_for_outbound_sockets=127.28.47.130
--tserver_master_addrs=127.28.47.190:42635
--webserver_port=37893
--webserver_interface=127.28.47.130
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab
--principal=oryx/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.47.190:42635,127.28.47.189:32861
--tserver_master_addrs=127.28.47.190:42635,127.28.47.189:32861,127.28.47.188:35257 with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:09:47.008796 4238 heartbeater.cc:503] Master 127.28.47.190:42635 was elected leader, sending a full tablet report...
Aug 11 22:09:47 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486186, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.189@KRBTEST.COM
Aug 11 22:09:47 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486186, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.188@KRBTEST.COM
Aug 11 22:09:47 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486186, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:47.016304 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.985533 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:58421 (local address 127.28.47.190:42635)
0811 22:09:46.985682 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.985688 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.987349 (+ 1661us) server_negotiation.cc:407] Connection header received
0811 22:09:46.999425 (+ 12076us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.999429 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.999495 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.999550 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.002274 (+ 2724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.003200 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.007377 (+ 4177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.007852 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.007947 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.012275 (+ 4328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.012280 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.012283 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.012318 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.014172 (+ 1854us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.014623 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.014627 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.014629 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.014810 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.015140 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.015143 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.015145 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.015390 (+ 245us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.015484 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.015975 (+ 491us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.016122 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:09:47.016427 4562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.985704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:47381 (local address 127.28.47.189:32861)
0811 22:09:46.985812 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:46.985818 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:46.987511 (+ 1693us) server_negotiation.cc:407] Connection header received
0811 22:09:46.999210 (+ 11699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:46.999215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:46.999283 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:46.999338 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.002664 (+ 3326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.003479 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.005468 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.005838 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.005925 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.010664 (+ 4739us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.010669 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.010672 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.010715 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.012831 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.013676 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.013683 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.013687 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.013794 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.014680 (+ 886us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.014686 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.014689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.015010 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.015114 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.016012 (+ 898us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.016179 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 22:09:47.017536 4852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.985225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.129:58421)
0811 22:09:46.986859 (+ 1634us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.986908 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.998842 (+ 11934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.999783 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.999795 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.000923 (+ 1128us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:47.002046 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.002070 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.003547 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.003553 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.006339 (+ 2786us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.006354 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.009053 (+ 2699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.009058 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.009263 (+ 205us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.010337 (+ 1074us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:47.010353 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:47.012090 (+ 1737us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:47.014295 (+ 2205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.014301 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.014306 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.014504 (+ 198us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.014919 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.014921 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.014923 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.015024 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.015536 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:47.015539 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:47.015711 (+ 172us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.016628 (+ 917us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.016789 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1301,"mutex_wait_us":91,"spinlock_wait_cycles":8960,"thread_start_us":653,"threads_started":1}
I20260811 22:09:47.017467 4853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.986979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:35257 (local address 127.28.47.129:45659)
0811 22:09:46.998358 (+ 11379us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.998377 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.999148 (+ 771us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:47.001176 (+ 2028us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:47.001179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.001477 (+ 298us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:47.002043 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.002066 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.003718 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.003721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.005367 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.005382 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.007164 (+ 1782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.007172 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.007358 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.008078 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:47.008096 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:47.011139 (+ 3043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:47.013952 (+ 2813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.013955 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.013957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.014272 (+ 315us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.014893 (+ 621us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.014900 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.014902 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.015024 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.015509 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:47.015513 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:47.015677 (+ 164us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.015881 (+ 204us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.016281 (+ 400us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":11271,"mutex_wait_us":127,"spinlock_wait_cycles":6272,"thread_start_us":174,"threads_started":1}
I20260811 22:09:47.018141 4856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.987108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:45659 (local address 127.28.47.188:35257)
0811 22:09:47.000825 (+ 13717us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.000831 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.000850 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:09:47.000908 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.000912 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.000992 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.001624 (+ 632us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.002227 (+ 603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.003536 (+ 1309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.006527 (+ 2991us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.006994 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.008843 (+ 1849us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.011334 (+ 2491us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.011340 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.011343 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.011378 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.013806 (+ 2428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.014670 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.014676 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.014680 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.014770 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.015142 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.015145 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.015147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.015390 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.017600 (+ 2210us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.017712 (+ 112us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.017885 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":13611,"thread_start_us":142,"threads_started":1}
I20260811 22:09:47.018666 4854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:46.985366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:32861 (local address 127.28.47.129:47381)
0811 22:09:46.987377 (+ 2011us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:46.987389 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:46.998842 (+ 11453us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:46.999783 (+ 941us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:46.999795 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.000791 (+ 996us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:47.002497 (+ 1706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.002508 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.003636 (+ 1128us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.003639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.005291 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.005308 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.006142 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.006145 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.006367 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.007167 (+ 800us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:47.007193 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:47.010341 (+ 3148us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:47.013013 (+ 2672us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.013024 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.013050 (+ 26us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.013533 (+ 483us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.013946 (+ 413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.013952 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.013956 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.014545 (+ 589us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.015202 (+ 657us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:47.015212 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:47.015653 (+ 441us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.017144 (+ 1491us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.017347 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1945,"mutex_wait_us":128,"spinlock_wait_cycles":26368,"thread_start_us":579,"threads_started":1}
I20260811 22:09:47.019707 4850 heartbeater.cc:347] Connected to a master server at 127.28.47.189:32861
I20260811 22:09:47.019819 4848 heartbeater.cc:347] Connected to a master server at 127.28.47.188:35257
I20260811 22:09:47.020010 4849 heartbeater.cc:347] Connected to a master server at 127.28.47.190:42635
I20260811 22:09:47.020056 4848 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:47.020059 4850 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:47.020117 4849 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:47.020874 4848 heartbeater.cc:511] Master 127.28.47.188:35257 requested a full tablet report, sending...
I20260811 22:09:47.020893 4849 heartbeater.cc:511] Master 127.28.47.190:42635 requested a full tablet report, sending...
I20260811 22:09:47.020874 4850 heartbeater.cc:511] Master 127.28.47.189:32861 requested a full tablet report, sending...
I20260811 22:09:47.022528 3698 ts_manager.cc:194] Re-registered known tserver with Master: 0f37525d62de4040b227093cc227011a (127.28.47.129:39521)
I20260811 22:09:47.022853 4627 ts_manager.cc:194] Registered new tserver with Master: 0f37525d62de4040b227093cc227011a (127.28.47.129:39521)
I20260811 22:09:47.023324 3698 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:58421
I20260811 22:09:47.024433 4419 ts_manager.cc:194] Registered new tserver with Master: 0f37525d62de4040b227093cc227011a (127.28.47.129:39521)
W20260811 22:09:47.139196 4857 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:47.139565 4857 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:47.139654 4857 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:47.144311 4857 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:47.144412 4857 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:47.144511 4857 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:47.151063 4857 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab
--principal=oryx/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:37765
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=37893
--tserver_master_addrs=127.28.47.190:42635,127.28.47.189:32861,127.28.47.188:35257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4857
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:47.152369 4857 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:47.153613 4857 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:47.164916 4863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:47.164924 4864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:47.165047 4866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:47.165652 4857 server_base.cc:1034] running on GCE node
I20260811 22:09:47.166205 4857 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:47.167169 4857 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:47.168349 4857 hybrid_clock.cc:648] HybridClock initialized: now 1786486187168335 us; error 40 us; skew 500 ppm
Aug 11 22:09:47 dist-test-slave-vxj2 krb5kdc[3643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486187, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:47.173090 4857 init.cc:377] Logged in from keytab as oryx/127.28.47.130@KRBTEST.COM (short username oryx)
I20260811 22:09:47.174266 4857 webserver.cc:460] Webserver started at http://127.28.47.130:37893/ using document root <none> and password file <none>
I20260811 22:09:47.174913 4857 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:47.175037 4857 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:47.180207 4857 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:09:47.182781 4874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:47.184255 4857 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:09:47.184449 4857 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "f99e42e45a6e4a668e580c173dbd1d61"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "032640de377884a139566d34bb28e65d"
server_key_iv: "42745e6385b771f1f987241ed8d13cf3"
server_key_version: "encryptionkey@0"
I20260811 22:09:47.185199 4857 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:47.237537 4857 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:47.241799 4857 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:47.242105 4857 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:47.242991 4857 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:47.244056 4857 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:47.244133 4857 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:47.244210 4857 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:47.244257 4857 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:47.257884 4857 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:37765
I20260811 22:09:47.257901 5011 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:37765 every 8 connection(s)
I20260811 22:09:47.259217 4857 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:47.266199 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 4857
I20260811 22:09:47.266300 4362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.261451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:35257 (local address 127.0.0.1:52416)
0811 22:09:47.261652 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:47.261667 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:47.261827 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:47.262346 (+ 519us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:47.262351 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.262378 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:47.262492 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.262500 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.263565 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.263571 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.264647 (+ 1076us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.264656 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.265788 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.265793 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.265917 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.265956 (+ 39us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.266042 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.266144 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115}
I20260811 22:09:47.266399 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 4081
Aug 11 22:09:47 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486187, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.188@KRBTEST.COM
Aug 11 22:09:47 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486187, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:47.296948 5017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.261551 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:48515 (local address 127.28.47.188:35257)
0811 22:09:47.262238 (+ 687us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.262243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.263820 (+ 1577us) server_negotiation.cc:407] Connection header received
0811 22:09:47.264656 (+ 836us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.264660 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.264717 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.264813 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.281659 (+ 16846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.282456 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.286434 (+ 3978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.286847 (+ 413us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.286936 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.291782 (+ 4846us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.291786 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.291789 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.291821 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.293963 (+ 2142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.294813 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.294817 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.294819 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.294906 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.295404 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.295407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.295410 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.295667 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.295753 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.296626 (+ 873us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.296778 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":615,"thread_start_us":102,"threads_started":1}
I20260811 22:09:47.296943 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.263432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:44607 (local address 127.28.47.190:42635)
0811 22:09:47.263554 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.263558 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.264016 (+ 458us) server_negotiation.cc:407] Connection header received
0811 22:09:47.264647 (+ 631us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.264650 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.264709 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.264764 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.284690 (+ 19926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.285468 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.286713 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.287026 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.287104 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.291778 (+ 4674us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.291783 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.291786 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.291822 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.294175 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.294801 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.294805 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.294807 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.294906 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.295404 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.295407 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.295409 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.295673 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.295748 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.296626 (+ 878us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.296769 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:09:47.299003 5019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.263337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.130:44607)
0811 22:09:47.263914 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:47.263928 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:47.264406 (+ 478us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:47.265243 (+ 837us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:47.265260 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.265974 (+ 714us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:47.281367 (+ 15393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.281400 (+ 33us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.285649 (+ 4249us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.285652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.286582 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.286592 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.287277 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.287280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.287505 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.288503 (+ 998us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:47.288515 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:47.291506 (+ 2991us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:47.294331 (+ 2825us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.294334 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.294336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.294668 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.295028 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.295031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.295033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.295281 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.295842 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:47.295850 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:47.296299 (+ 449us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.297366 (+ 1067us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.297775 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":524,"mutex_wait_us":133,"spinlock_wait_cycles":70912,"thread_start_us":117,"threads_started":1}
I20260811 22:09:47.299003 5020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.262063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:35257 (local address 127.28.47.130:48515)
0811 22:09:47.263703 (+ 1640us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:47.263717 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:47.264401 (+ 684us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:47.265243 (+ 842us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:47.265254 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.281308 (+ 16054us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:47.281465 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.281473 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.282768 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.282866 (+ 98us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.286263 (+ 3397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.286276 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.287205 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.287211 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.287507 (+ 296us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.288425 (+ 918us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:47.288463 (+ 38us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:47.291506 (+ 3043us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:47.294144 (+ 2638us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.294153 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.294172 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.294668 (+ 496us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.295028 (+ 360us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.295031 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.295033 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.295281 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.295842 (+ 561us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:47.295850 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:47.296299 (+ 449us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.297669 (+ 1370us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.297829 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1548,"mutex_wait_us":187,"spinlock_wait_cycles":37120,"thread_start_us":210,"threads_started":1}
I20260811 22:09:47.300787 5013 heartbeater.cc:347] Connected to a master server at 127.28.47.190:42635
I20260811 22:09:47.300880 5012 heartbeater.cc:347] Connected to a master server at 127.28.47.188:35257
I20260811 22:09:47.301061 5012 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:47.301061 5013 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:47.301786 5013 heartbeater.cc:511] Master 127.28.47.190:42635 requested a full tablet report, sending...
I20260811 22:09:47.301786 5012 heartbeater.cc:511] Master 127.28.47.188:35257 requested a full tablet report, sending...
I20260811 22:09:47.301935 4856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.261514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52416 (local address 127.28.47.188:35257)
0811 22:09:47.261674 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.261679 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.261930 (+ 251us) server_negotiation.cc:407] Connection header received
0811 22:09:47.262002 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.262004 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.262075 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.262198 (+ 123us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:47.262645 (+ 447us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.263428 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.264794 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.265515 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.265633 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.265838 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.301600 (+ 35762us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.301756 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:09:47.302882 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:39211
--local_ip_for_outbound_sockets=127.28.47.131
--tserver_master_addrs=127.28.47.190:42635
--webserver_port=45567
--webserver_interface=127.28.47.131
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab
--principal=oryx/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39537
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.28.47.190:42635,127.28.47.189:32861
--tserver_master_addrs=127.28.47.190:42635,127.28.47.189:32861,127.28.47.188:35257 with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:09:47.302798 4649 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260811 22:09:47.303153 3698 ts_manager.cc:194] Re-registered known tserver with Master: f99e42e45a6e4a668e580c173dbd1d61 (127.28.47.130:37765)
I20260811 22:09:47.303471 4627 ts_manager.cc:194] Registered new tserver with Master: f99e42e45a6e4a668e580c173dbd1d61 (127.28.47.130:37765)
I20260811 22:09:47.303570 4686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bfa9d50481124d2e9f97ffe5a3ddc26f. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } } }
I20260811 22:09:47.303747 4686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: This master's current role is: LEARNER
I20260811 22:09:47.304029 3698 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:44607
Aug 11 22:09:47 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486187, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.189@KRBTEST.COM
I20260811 22:09:47.314167 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.262063 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:32861 (local address 127.28.47.130:45137)
0811 22:09:47.263194 (+ 1131us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:47.263256 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:47.264387 (+ 1131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:47.265243 (+ 856us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:47.265254 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.265977 (+ 723us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:47.304970 (+ 38993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.304987 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.306210 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.306214 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.306830 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.306840 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.307411 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.307415 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.307514 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.308148 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:47.308161 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:47.310315 (+ 2154us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:47.312332 (+ 2017us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.312336 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.312339 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.312567 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.312876 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.312879 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.312881 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.312974 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.313528 (+ 554us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:47.313531 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:47.313650 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.313775 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.313971 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":906,"mutex_wait_us":38407,"thread_start_us":186,"threads_started":1}
I20260811 22:09:47.314149 4562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.261827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:45137 (local address 127.28.47.189:32861)
0811 22:09:47.262026 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.262031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.263352 (+ 1321us) server_negotiation.cc:407] Connection header received
0811 22:09:47.264658 (+ 1306us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.264661 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.264717 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.264793 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.305298 (+ 40505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.306057 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.306991 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.307244 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.307387 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.310506 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.310511 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.310513 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.310554 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.312167 (+ 1613us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.312684 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.312693 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.312694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.312777 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.313072 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.313076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.313077 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.313395 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.313545 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.313836 (+ 291us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.313968 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:09:47.315479 5014 heartbeater.cc:347] Connected to a master server at 127.28.47.189:32861
I20260811 22:09:47.315650 5014 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:47.315960 5014 heartbeater.cc:511] Master 127.28.47.189:32861 requested a full tablet report, sending...
I20260811 22:09:47.317065 4419 ts_manager.cc:194] Registered new tserver with Master: f99e42e45a6e4a668e580c173dbd1d61 (127.28.47.130:37765)
W20260811 22:09:47.462087 5021 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:47.462381 5021 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:47.462457 5021 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:47.466790 5021 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:47.466861 5021 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:47.466992 5021 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:47.473377 5021 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39537
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab
--principal=oryx/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:39211
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=45567
--tserver_master_addrs=127.28.47.190:42635,127.28.47.189:32861,127.28.47.188:35257
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5021
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:47.474697 5021 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:47.475952 5021 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:47.488348 5028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:47.489352 5027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:47.489377 5030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:47.489740 5021 server_base.cc:1034] running on GCE node
I20260811 22:09:47.490434 5021 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:47.491405 5021 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:47.492636 5021 hybrid_clock.cc:648] HybridClock initialized: now 1786486187492606 us; error 52 us; skew 500 ppm
Aug 11 22:09:47 dist-test-slave-vxj2 krb5kdc[3643](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486187, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:47.496946 5021 init.cc:377] Logged in from keytab as oryx/127.28.47.131@KRBTEST.COM (short username oryx)
I20260811 22:09:47.498464 5021 webserver.cc:460] Webserver started at http://127.28.47.131:45567/ using document root <none> and password file <none>
I20260811 22:09:47.499348 5021 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:47.499477 5021 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:47.506285 5021 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:09:47.509688 5038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:47.511619 5021 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:09:47.511812 5021 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "87d7918cb98a4bdf8fb75f538a3ad408"
format_stamp: "Formatted at 2026-08-11 22:09:45 on dist-test-slave-vxj2"
server_key: "2f8bc81a73e93705d7510c8014463493"
server_key_iv: "d019f49c81238aaf85b698ba6371f141"
server_key_version: "encryptionkey@0"
I20260811 22:09:47.512821 5021 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:47.546351 5021 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:47.551205 5021 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:47.551507 5021 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:47.552597 5021 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:47.554158 5021 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:47.554241 5021 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:47.554311 5021 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:47.554334 5021 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:47.572026 5021 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:39211
I20260811 22:09:47.572057 5176 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:39211 every 8 connection(s)
I20260811 22:09:47.573592 5021 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:47.575074 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 5021
Aug 11 22:09:47 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486187, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:47.590443 5017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.578155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52430 (local address 127.28.47.188:35257)
0811 22:09:47.578343 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.578347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.578597 (+ 250us) server_negotiation.cc:407] Connection header received
0811 22:09:47.578818 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.578824 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.578888 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.578948 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.580110 (+ 1162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.580887 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.582280 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.582626 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.582716 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.584874 (+ 2158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.584878 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.584881 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.584911 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.587963 (+ 3052us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.588504 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.588511 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.588515 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.588616 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.588964 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.588970 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.588974 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.589283 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.589401 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.589834 (+ 433us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.589982 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100}
I20260811 22:09:47.591390 5193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.578392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41652 (local address 127.28.47.190:42635)
0811 22:09:47.578665 (+ 273us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.578670 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.578687 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:09:47.578771 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.578774 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.578828 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.580005 (+ 1177us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.580107 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.580954 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.582375 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.582733 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.582863 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.585421 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.585425 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.585428 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.585452 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.588256 (+ 2804us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.588998 (+ 742us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.589004 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.589006 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.589101 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.589597 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.589604 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.589606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.589852 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.589985 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.590886 (+ 901us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.591134 (+ 248us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":195,"thread_start_us":98,"threads_started":1}
Aug 11 22:09:47 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486187, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.189@KRBTEST.COM
Aug 11 22:09:47 dist-test-slave-vxj2 krb5kdc[3643](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486187, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.188@KRBTEST.COM
I20260811 22:09:47.597018 4562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.576020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:44483 (local address 127.28.47.189:32861)
0811 22:09:47.576287 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.576292 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.578223 (+ 1931us) server_negotiation.cc:407] Connection header received
0811 22:09:47.579392 (+ 1169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.579398 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.579464 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.579528 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.581926 (+ 2398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.582769 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.586269 (+ 3500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.586593 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.586672 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.592888 (+ 6216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.592892 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.592895 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.592921 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.594900 (+ 1979us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.595449 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.595452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.595453 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.595540 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.595864 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.595868 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.595869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.596103 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.596191 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.596742 (+ 551us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.596864 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":179}
I20260811 22:09:47.597067 3746 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.576152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:39919 (local address 127.28.47.190:42635)
0811 22:09:47.576385 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.576389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.579321 (+ 2932us) server_negotiation.cc:407] Connection header received
0811 22:09:47.579583 (+ 262us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.579587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.579648 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.579710 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.581892 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.582638 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.584655 (+ 2017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.585004 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.585111 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.592051 (+ 6940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.592055 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.592058 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.592081 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.594238 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.594993 (+ 755us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.594997 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.594999 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.595093 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.595601 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.595607 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.595612 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.595864 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.595955 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.596753 (+ 798us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.596888 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260811 22:09:47.599195 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.578702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58216 (local address 127.28.47.189:32861)
0811 22:09:47.580950 (+ 2248us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.580956 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.580971 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:09:47.581011 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.581015 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.581062 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.581144 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.583057 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.583875 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.586499 (+ 2624us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.586892 (+ 393us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.587011 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.589745 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.589751 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.589754 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.589801 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.592058 (+ 2257us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.593413 (+ 1355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.593419 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.593424 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.593530 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.593895 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.593898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.593900 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.598409 (+ 4509us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.598538 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.598889 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.599019 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2161,"thread_start_us":106,"threads_started":1}
I20260811 22:09:47.599560 4856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.578078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:33443 (local address 127.28.47.188:35257)
0811 22:09:47.578244 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.578248 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.585390 (+ 7142us) server_negotiation.cc:407] Connection header received
0811 22:09:47.585598 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.585602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.585661 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.585722 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.586475 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.587204 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.590157 (+ 2953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.590506 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.590587 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.594427 (+ 3840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.594431 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.594434 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.594460 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.596203 (+ 1743us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.596680 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.596684 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.596686 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.596783 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.597571 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.597575 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.597576 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.597842 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.597937 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.599252 (+ 1315us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.599395 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:09:47.599575 5187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.576386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:32861 (local address 127.28.47.131:44483)
0811 22:09:47.577821 (+ 1435us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:47.577884 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:47.579118 (+ 1234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:47.580013 (+ 895us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:47.580023 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.580647 (+ 624us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:47.581438 (+ 791us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.581461 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.584702 (+ 3241us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.584705 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.585618 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.585630 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.587076 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.587080 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.587185 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.590494 (+ 3309us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:47.590513 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:47.592718 (+ 2205us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:47.595069 (+ 2351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.595072 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.595076 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.595330 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.595656 (+ 326us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.595659 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.595661 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.595750 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.596278 (+ 528us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:47.596281 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:47.596448 (+ 167us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.597208 (+ 760us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.597652 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1179,"mutex_wait_us":5,"spinlock_wait_cycles":2176,"thread_start_us":179,"threads_started":1}
I20260811 22:09:47.599663 5186 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.576386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.131:39919)
0811 22:09:47.579228 (+ 2842us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:47.579243 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:47.579399 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:47.580014 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:47.580025 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.581654 (+ 1629us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:47.581782 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.581789 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.582931 (+ 1142us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.582935 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.584475 (+ 1540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.584488 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.585813 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.585817 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.586086 (+ 269us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.587653 (+ 1567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:47.587683 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:47.590154 (+ 2471us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:47.594391 (+ 4237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.594399 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.594421 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.594861 (+ 440us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.595216 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.595219 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.595223 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.595473 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.596035 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:47.596043 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:47.596447 (+ 404us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.597309 (+ 862us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.597753 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2730,"mutex_wait_us":826,"spinlock_wait_cycles":35840,"thread_start_us":213,"threads_started":1}
I20260811 22:09:47.596407 3698 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260811 22:09:47.600879 5178 heartbeater.cc:347] Connected to a master server at 127.28.47.190:42635
I20260811 22:09:47.600927 5179 heartbeater.cc:347] Connected to a master server at 127.28.47.189:32861
I20260811 22:09:47.601126 5179 heartbeater.cc:464] Registering TS with master...
W20260811 22:09:47.601158 3698 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:47.601838 5179 heartbeater.cc:511] Master 127.28.47.189:32861 requested a full tablet report, sending...
I20260811 22:09:47.601126 5178 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:47.602612 5178 heartbeater.cc:511] Master 127.28.47.190:42635 requested a full tablet report, sending...
I20260811 22:09:47.603080 5190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.577667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:35257 (local address 127.28.47.131:33443)
0811 22:09:47.585296 (+ 7629us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:47.585313 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:47.585473 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:47.585901 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:47.585904 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.586148 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:47.586328 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.586339 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.587521 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.587525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.588419 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.588432 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.591181 (+ 2749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.591190 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.591319 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.592620 (+ 1301us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:47.592637 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:47.594259 (+ 1622us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:47.596331 (+ 2072us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.596334 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.596336 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.596557 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.596899 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:47.596902 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:47.596904 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:47.596984 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:47.597999 (+ 1015us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:47.598002 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:47.598092 (+ 90us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.602607 (+ 4515us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.602840 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7540,"mutex_wait_us":1,"thread_start_us":84,"threads_started":1}
I20260811 22:09:47.603499 3697 ts_manager.cc:194] Re-registered known tserver with Master: 87d7918cb98a4bdf8fb75f538a3ad408 (127.28.47.131:39211)
I20260811 22:09:47.603550 4419 ts_manager.cc:194] Registered new tserver with Master: 87d7918cb98a4bdf8fb75f538a3ad408 (127.28.47.131:39211)
I20260811 22:09:47.603816 5177 heartbeater.cc:347] Connected to a master server at 127.28.47.188:35257
I20260811 22:09:47.603893 5177 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:47.604065 5177 heartbeater.cc:511] Master 127.28.47.188:35257 requested a full tablet report, sending...
I20260811 22:09:47.604305 3697 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:39919
I20260811 22:09:47.604746 4590 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: NON_VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: true } } }
I20260811 22:09:47.604897 4590 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:47.604907 4627 ts_manager.cc:194] Registered new tserver with Master: 87d7918cb98a4bdf8fb75f538a3ad408 (127.28.47.131:39211)
I20260811 22:09:47.608716 5196 raft_consensus.cc:1050] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f: attempting to promote NON_VOTER 70a6d0aaab4b4d6b8f6da676b5016064 to VOTER
I20260811 22:09:47.609124 5196 consensus_queue.cc:237] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } }
I20260811 22:09:47.610518 4649 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 1 LEARNER]: Refusing update from remote peer bfa9d50481124d2e9f97ffe5a3ddc26f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 22:09:47.610985 4590 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [LEADER]: Connected to new peer: Peer: permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 22:09:47.611840 4431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 FOLLOWER]: Refusing update from remote peer bfa9d50481124d2e9f97ffe5a3ddc26f: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260811 22:09:47.612258 4590 consensus_queue.cc:1035] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [LEADER]: Connected to new peer: Peer: permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260811 22:09:47.613610 5203 raft_consensus.cc:2932] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 70a6d0aaab4b4d6b8f6da676b5016064 (127.28.47.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } } }
I20260811 22:09:47.614094 4649 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 70a6d0aaab4b4d6b8f6da676b5016064 (127.28.47.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } } }
I20260811 22:09:47.614662 4431 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 70a6d0aaab4b4d6b8f6da676b5016064 (127.28.47.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } } }
I20260811 22:09:47.615834 5196 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } } }
I20260811 22:09:47.615988 5196 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:47.616504 5203 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } } }
I20260811 22:09:47.616621 5203 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:47.616564 5196 sys_catalog.cc:455] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } } }
I20260811 22:09:47.616674 5196 sys_catalog.cc:458] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:47.616912 5207 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } } }
I20260811 22:09:47.617136 5207 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:09:47.617624 4686 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } } }
I20260811 22:09:47.617751 4686 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:09:47.627837 5212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.621468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:39521 (local address 127.0.0.1:45538)
0811 22:09:47.621920 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:47.621935 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:47.622026 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:47.623372 (+ 1346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:47.623375 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.623396 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:47.623504 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.623514 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.625097 (+ 1583us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.625101 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.626266 (+ 1165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.626276 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.627369 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.627373 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.627467 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.627495 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.627569 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.627658 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"thread_start_us":178,"threads_started":1}
I20260811 22:09:47.627847 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.621668 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45538 (local address 127.28.47.129:39521)
0811 22:09:47.622326 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.622333 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.622358 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:09:47.623044 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.623051 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.623217 (+ 166us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.623382 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:47.623653 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.624860 (+ 1207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.626442 (+ 1582us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.627221 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.627365 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.627550 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.627611 (+ 61us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.627719 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":549,"thread_start_us":112,"threads_started":1}
I20260811 22:09:47.628187 5214 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.621937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:37765 (local address 127.0.0.1:55660)
0811 22:09:47.622340 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:47.622354 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:47.622490 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:47.623567 (+ 1077us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:47.623571 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.623593 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:47.623681 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.623688 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.625419 (+ 1731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.625424 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.626300 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.626310 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.627722 (+ 1412us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.627727 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.627854 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.627884 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.627941 (+ 57us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.628022 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":5,"thread_start_us":234,"threads_started":1}
I20260811 22:09:47.628907 5215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.622670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55660 (local address 127.28.47.130:37765)
0811 22:09:47.623105 (+ 435us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.623129 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.623152 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:09:47.623224 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.623229 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.623423 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.623587 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:47.623825 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.625269 (+ 1444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.626473 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.627552 (+ 1079us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.628430 (+ 878us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.628585 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.628632 (+ 47us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.628704 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":146,"threads_started":1}
I20260811 22:09:47.629556 4362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.620702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:39211 (local address 127.0.0.1:40290)
0811 22:09:47.620839 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:47.620860 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:47.620961 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:47.622628 (+ 1667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:47.622631 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.622652 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:47.622787 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.622818 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.624590 (+ 1772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.624594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.626638 (+ 2044us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.626648 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.629099 (+ 2451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.629104 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.629242 (+ 138us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.629269 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.629357 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.629432 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":49,"mutex_wait_us":759}
I20260811 22:09:47.630016 5211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.621059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40290 (local address 127.28.47.131:39211)
0811 22:09:47.621659 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.621675 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.621705 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:09:47.621810 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.621817 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.622154 (+ 337us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.622357 (+ 203us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:47.623057 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.624424 (+ 1367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.626818 (+ 2394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.627760 (+ 942us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.627867 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.628018 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.629778 (+ 1760us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.629897 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":458,"thread_start_us":167,"threads_started":1}
I20260811 22:09:47.631692 4768 tablet_service.cc:1467] Processing CreateTablet for tablet 8ff3c8cda1d44d978de43f538454dfdd (DEFAULT_TABLE table=test-table [id=ee18ec6b3edf474895af0474eba7b584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:47.631654 4931 tablet_service.cc:1467] Processing CreateTablet for tablet 8ff3c8cda1d44d978de43f538454dfdd (DEFAULT_TABLE table=test-table [id=ee18ec6b3edf474895af0474eba7b584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:47.632385 5094 tablet_service.cc:1467] Processing CreateTablet for tablet 8ff3c8cda1d44d978de43f538454dfdd (DEFAULT_TABLE table=test-table [id=ee18ec6b3edf474895af0474eba7b584]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:47.632915 4931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ff3c8cda1d44d978de43f538454dfdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:47.632915 4768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ff3c8cda1d44d978de43f538454dfdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:47.633478 5094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8ff3c8cda1d44d978de43f538454dfdd. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:47.644340 5219 tablet_bootstrap.cc:492] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408: Bootstrap starting.
I20260811 22:09:47.644454 5220 tablet_bootstrap.cc:492] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a: Bootstrap starting.
I20260811 22:09:47.644502 5221 tablet_bootstrap.cc:492] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61: Bootstrap starting.
I20260811 22:09:47.647441 5219 tablet_bootstrap.cc:654] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:47.647547 5221 tablet_bootstrap.cc:654] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:47.647557 5220 tablet_bootstrap.cc:654] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:47.648979 5219 log.cc:824] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:47.649073 5220 log.cc:824] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:47.649072 5221 log.cc:824] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:47.651688 5219 tablet_bootstrap.cc:492] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408: No bootstrap required, opened a new log
I20260811 22:09:47.651718 5221 tablet_bootstrap.cc:492] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61: No bootstrap required, opened a new log
I20260811 22:09:47.651747 5220 tablet_bootstrap.cc:492] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a: No bootstrap required, opened a new log
I20260811 22:09:47.651963 5221 ts_tablet_manager.cc:1397] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 22:09:47.651962 5220 ts_tablet_manager.cc:1397] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 22:09:47.651964 5219 ts_tablet_manager.cc:1397] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260811 22:09:47.658218 5219 raft_consensus.cc:348] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:47.658545 5219 raft_consensus.cc:374] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:47.658663 5219 raft_consensus.cc:729] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87d7918cb98a4bdf8fb75f538a3ad408, State: Initialized, Role: FOLLOWER
I20260811 22:09:47.659207 5219 consensus_queue.cc:260] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:47.659427 5220 raft_consensus.cc:348] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:47.659721 5220 raft_consensus.cc:374] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:47.659796 5220 raft_consensus.cc:729] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f37525d62de4040b227093cc227011a, State: Initialized, Role: FOLLOWER
I20260811 22:09:47.659793 5221 raft_consensus.cc:348] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:47.660053 5221 raft_consensus.cc:374] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:47.660148 5221 raft_consensus.cc:729] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99e42e45a6e4a668e580c173dbd1d61, State: Initialized, Role: FOLLOWER
I20260811 22:09:47.660365 5178 heartbeater.cc:503] Master 127.28.47.190:42635 was elected leader, sending a full tablet report...
I20260811 22:09:47.660490 5220 consensus_queue.cc:260] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:47.661249 5219 ts_tablet_manager.cc:1428] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408: Time spent starting tablet: real 0.009s user 0.009s sys 0.000s
I20260811 22:09:47.661624 5220 ts_tablet_manager.cc:1428] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a: Time spent starting tablet: real 0.010s user 0.009s sys 0.000s
I20260811 22:09:47.662433 5221 consensus_queue.cc:260] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:47.663831 5221 ts_tablet_manager.cc:1428] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61: Time spent starting tablet: real 0.012s user 0.005s sys 0.004s
I20260811 22:09:47.664573 5228 raft_consensus.cc:482] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:47.664794 5228 raft_consensus.cc:504] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:47.665189 5013 heartbeater.cc:503] Master 127.28.47.190:42635 was elected leader, sending a full tablet report...
I20260811 22:09:47.666936 4849 heartbeater.cc:503] Master 127.28.47.190:42635 was elected leader, sending a full tablet report...
I20260811 22:09:47.667289 5228 leader_election.cc:288] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 87d7918cb98a4bdf8fb75f538a3ad408 (127.28.47.131:39211), 0f37525d62de4040b227093cc227011a (127.28.47.129:39521)
I20260811 22:09:47.672564 5018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.666975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:39211 (local address 127.28.47.130:37543)
0811 22:09:47.667102 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:47.667119 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:47.667203 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:47.668213 (+ 1010us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:47.668218 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.668238 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:47.668477 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.668485 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.669461 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.669464 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.670842 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.670852 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.671943 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.671947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.672123 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.672150 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.672236 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.672310 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41,"mutex_wait_us":1}
I20260811 22:09:47.673014 5211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.667682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:37543 (local address 127.28.47.131:39211)
0811 22:09:47.667864 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.667869 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.667889 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:09:47.667935 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.667938 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.667990 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.668083 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:47.668649 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.669310 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.671003 (+ 1693us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.671776 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.671906 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.671965 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.672798 (+ 833us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.672889 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:09:47.674181 5019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.667583 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:39521 (local address 127.28.47.130:60357)
0811 22:09:47.668692 (+ 1109us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:47.668709 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:47.668782 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:47.669606 (+ 824us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:47.669609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:47.669628 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:47.669853 (+ 225us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.669862 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.670871 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.670876 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.672185 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:47.672196 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.673638 (+ 1442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.673644 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:47.673845 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.673874 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:09:47.673972 (+ 98us) client_negotiation.cc:240] Negotiation successful
0811 22:09:47.674068 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1012}
I20260811 22:09:47.674703 5213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.669281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:60357 (local address 127.28.47.129:39521)
0811 22:09:47.669400 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.669404 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.669416 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:09:47.669453 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.669455 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.669498 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.669594 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:47.670013 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.670738 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.672332 (+ 1594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.672912 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.673047 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.673113 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.674435 (+ 1322us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.674542 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:09:47.674693 5121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ff3c8cda1d44d978de43f538454dfdd" candidate_uuid: "f99e42e45a6e4a668e580c173dbd1d61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" is_pre_election: true
I20260811 22:09:47.675078 5121 raft_consensus.cc:2454] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99e42e45a6e4a668e580c173dbd1d61 in term 0.
I20260811 22:09:47.675669 4877 leader_election.cc:302] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87d7918cb98a4bdf8fb75f538a3ad408, f99e42e45a6e4a668e580c173dbd1d61; no voters:
I20260811 22:09:47.676224 5230 raft_consensus.cc:2781] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:47.676507 5230 raft_consensus.cc:482] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:47.676605 5230 raft_consensus.cc:3037] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:47.679096 5230 raft_consensus.cc:504] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:47.679276 4790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ff3c8cda1d44d978de43f538454dfdd" candidate_uuid: "f99e42e45a6e4a668e580c173dbd1d61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f37525d62de4040b227093cc227011a" is_pre_election: true
I20260811 22:09:47.679631 4790 raft_consensus.cc:2454] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99e42e45a6e4a668e580c173dbd1d61 in term 0.
I20260811 22:09:47.680265 5230 leader_election.cc:288] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [CANDIDATE]: Term 1 election: Requested vote from peers 87d7918cb98a4bdf8fb75f538a3ad408 (127.28.47.131:39211), 0f37525d62de4040b227093cc227011a (127.28.47.129:39521)
I20260811 22:09:47.680250 5121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ff3c8cda1d44d978de43f538454dfdd" candidate_uuid: "f99e42e45a6e4a668e580c173dbd1d61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87d7918cb98a4bdf8fb75f538a3ad408"
I20260811 22:09:47.680400 5121 raft_consensus.cc:3037] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:47.680887 4790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8ff3c8cda1d44d978de43f538454dfdd" candidate_uuid: "f99e42e45a6e4a668e580c173dbd1d61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f37525d62de4040b227093cc227011a"
I20260811 22:09:47.681038 4790 raft_consensus.cc:3037] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:47.682667 5121 raft_consensus.cc:2454] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99e42e45a6e4a668e580c173dbd1d61 in term 1.
I20260811 22:09:47.682979 4877 leader_election.cc:302] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87d7918cb98a4bdf8fb75f538a3ad408, f99e42e45a6e4a668e580c173dbd1d61; no voters:
I20260811 22:09:47.683254 4790 raft_consensus.cc:2454] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99e42e45a6e4a668e580c173dbd1d61 in term 1.
I20260811 22:09:47.683214 5230 raft_consensus.cc:2781] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:47.683370 5230 raft_consensus.cc:686] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [term 1 LEADER]: Becoming Leader. State: Replica: f99e42e45a6e4a668e580c173dbd1d61, State: Running, Role: LEADER
I20260811 22:09:47.683845 5230 consensus_queue.cc:237] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:47.687463 3697 catalog_manager.cc:5495] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99e42e45a6e4a668e580c173dbd1d61 (127.28.47.130). New cstate: current_term: 1 leader_uuid: "f99e42e45a6e4a668e580c173dbd1d61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } health_report { overall_health: HEALTHY } } }
I20260811 22:09:47.711275 5215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.707332 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55672 (local address 127.28.47.130:37765)
0811 22:09:47.707498 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.707504 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.707520 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:09:47.707660 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.707663 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.707721 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.707843 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:47.708107 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.708626 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.709681 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.709908 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.710020 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.710378 (+ 358us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:47.710839 (+ 461us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:47.710979 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.711047 (+ 68us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.711117 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:09:47.718397 4790 raft_consensus.cc:1261] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a [term 1 FOLLOWER]: Refusing update from remote peer f99e42e45a6e4a668e580c173dbd1d61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:47.718398 5121 raft_consensus.cc:1261] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408 [term 1 FOLLOWER]: Refusing update from remote peer f99e42e45a6e4a668e580c173dbd1d61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:47.719269 5230 consensus_queue.cc:1035] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:47.719461 5228 consensus_queue.cc:1035] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:09:47.735569 4851 tablet.cc:2369] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:09:47.744117 3698 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:47.744347 3698 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:47.749967 3698 catalog_manager.cc:5782] T 00000000000000000000000000000000 P bfa9d50481124d2e9f97ffe5a3ddc26f: Sending DeleteTablet for 3 replicas of tablet 8ff3c8cda1d44d978de43f538454dfdd
I20260811 22:09:47.751286 4931 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ff3c8cda1d44d978de43f538454dfdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:47 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:55660
I20260811 22:09:47.751428 4768 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ff3c8cda1d44d978de43f538454dfdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:47 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:45538
I20260811 22:09:47.751955 5094 tablet_service.cc:1514] Processing DeleteTablet for tablet 8ff3c8cda1d44d978de43f538454dfdd with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:47 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:40290
I20260811 22:09:47.752601 5246 tablet_replica.cc:330] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61: stopping tablet replica
I20260811 22:09:47.753085 5246 raft_consensus.cc:2229] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:09:47.753650 5246 raft_consensus.cc:2258] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:47.755548 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 3658
I20260811 22:09:47.760841 5246 ts_tablet_manager.cc:1905] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:47.766340 5246 ts_tablet_manager.cc:1918] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:47.766520 5246 log.cc:1196] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/8ff3c8cda1d44d978de43f538454dfdd
I20260811 22:09:47.766913 5246 ts_tablet_manager.cc:1939] T 8ff3c8cda1d44d978de43f538454dfdd P f99e42e45a6e4a668e580c173dbd1d61: Deleting consensus metadata
I20260811 22:09:47.768630 5247 tablet_replica.cc:330] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a: stopping tablet replica
I20260811 22:09:47.769091 5247 raft_consensus.cc:2229] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:47.769546 5247 raft_consensus.cc:2258] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:47.770289 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.753544 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58226 (local address 127.28.47.189:32861)
0811 22:09:47.753679 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.753684 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.753996 (+ 312us) server_negotiation.cc:407] Connection header received
0811 22:09:47.754370 (+ 374us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.754374 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.754443 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.754518 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.759094 (+ 4576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.760034 (+ 940us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.761882 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.762355 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.762468 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.765866 (+ 3398us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.765871 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.765874 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.765913 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.768038 (+ 2125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.768672 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.768675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.768677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.768772 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.769106 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.769109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.769110 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.769428 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.769594 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.769972 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.770120 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:09:47.771651 5247 ts_tablet_manager.cc:1905] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 22:09:47.774320 4879 connection.cc:535] server connection from 127.0.0.1:55660 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:09:47.775219 5250 tablet_replica.cc:330] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408: stopping tablet replica
I20260811 22:09:47.775653 5250 raft_consensus.cc:2229] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:47.776144 5250 raft_consensus.cc:2258] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:47.777770 5247 ts_tablet_manager.cc:1918] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:47.777892 5247 log.cc:1196] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-0/wal/wals/8ff3c8cda1d44d978de43f538454dfdd
I20260811 22:09:47.779814 5250 ts_tablet_manager.cc:1905] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:47.781664 4856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.754101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52440 (local address 127.28.47.188:35257)
0811 22:09:47.754217 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.754222 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.770999 (+ 16777us) server_negotiation.cc:407] Connection header received
0811 22:09:47.771209 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.771212 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.771268 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.771323 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.772399 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.773298 (+ 899us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.774632 (+ 1334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.774986 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.775066 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.777076 (+ 2010us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.777081 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.777084 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.777120 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.779326 (+ 2206us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.780014 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.780018 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.780020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.780123 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.780586 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.780589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.780594 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.780835 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.780941 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.781341 (+ 400us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.781499 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:09:47.783531 5250 ts_tablet_manager.cc:1918] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:47.783675 5250 log.cc:1196] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/8ff3c8cda1d44d978de43f538454dfdd
I20260811 22:09:47.784111 5250 ts_tablet_manager.cc:1939] T 8ff3c8cda1d44d978de43f538454dfdd P 87d7918cb98a4bdf8fb75f538a3ad408: Deleting consensus metadata
I20260811 22:09:47.785590 5247 ts_tablet_manager.cc:1939] T 8ff3c8cda1d44d978de43f538454dfdd P 0f37525d62de4040b227093cc227011a: Deleting consensus metadata
W20260811 22:09:47.787002 4720 connection.cc:441] server connection from 127.0.0.1:45538 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:45538 (request call id 1) could send its response
W20260811 22:09:47.787667 5039 connection.cc:441] server connection from 127.0.0.1:40290 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:40290 (request call id 1) could send its response
I20260811 22:09:47.793483 5017 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.781290 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52446 (local address 127.28.47.188:35257)
0811 22:09:47.781423 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.781430 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.783087 (+ 1657us) server_negotiation.cc:407] Connection header received
0811 22:09:47.783299 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.783302 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.783364 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.783427 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.784758 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.785565 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.786851 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.787172 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.787259 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.789668 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.789672 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.789674 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.789701 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.791477 (+ 1776us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.791933 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.791937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.791940 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.792037 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.792380 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.792384 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.792386 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.792624 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.792725 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.793112 (+ 387us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.793297 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:09:47.793731 5194 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:47.779993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58228 (local address 127.28.47.189:32861)
0811 22:09:47.780289 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:47.780294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:47.782404 (+ 2110us) server_negotiation.cc:407] Connection header received
0811 22:09:47.782460 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:47.782463 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:47.782520 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:47.782598 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:47.785064 (+ 2466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.785902 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.787493 (+ 1591us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:47.787840 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:47.787916 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:47.790116 (+ 2200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:47.790120 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:47.790123 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:47.790153 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:47.791903 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.792352 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.792356 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.792358 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.792449 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:47.792729 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:47.792732 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:47.792734 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:47.792932 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:47.793064 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:47.793383 (+ 319us) server_negotiation.cc:299] Negotiation successful
0811 22:09:47.793498 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197}
I20260811 22:09:48.735402 5261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:48.734885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.131:51559)
0811 22:09:48.735179 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:48.735270 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":99,"threads_started":1}
W20260811 22:09:48.735800 5178 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:42635 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
I20260811 22:09:48.737192 5263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:48.736565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.130:58521)
0811 22:09:48.736912 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:48.737009 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":103,"threads_started":1}
W20260811 22:09:48.737577 5013 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:42635 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
I20260811 22:09:48.740424 5264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:48.739928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.129:44165)
0811 22:09:48.740219 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:48.740315 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":102,"threads_started":1}
W20260811 22:09:48.740754 4849 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:42635 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
I20260811 22:09:49.250168 5266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bfa9d50481124d2e9f97ffe5a3ddc26f)
I20260811 22:09:49.250397 5266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } }
I20260811 22:09:49.251103 5266 leader_election.cc:288] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfa9d50481124d2e9f97ffe5a3ddc26f (127.28.47.190:42635), 83b6f0f1668c4e0b8e60b5d3d4ea8a60 (127.28.47.189:32861)
I20260811 22:09:49.251093 5267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader bfa9d50481124d2e9f97ffe5a3ddc26f)
I20260811 22:09:49.251323 5267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } }
I20260811 22:09:49.251820 5269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.251302 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:42076)
0811 22:09:49.251621 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.251716 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":74,"threads_started":1}
W20260811 22:09:49.252035 4618 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
I20260811 22:09:49.252898 5267 leader_election.cc:288] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers bfa9d50481124d2e9f97ffe5a3ddc26f (127.28.47.190:42635), 70a6d0aaab4b4d6b8f6da676b5016064 (127.28.47.188:35257)
I20260811 22:09:49.253003 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.252635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:42082)
0811 22:09:49.252843 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.252915 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":60,"threads_started":1}
W20260811 22:09:49.253296 4392 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
I20260811 22:09:49.254165 5269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.253267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:42092)
0811 22:09:49.253663 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.253720 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":324}
W20260811 22:09:49.254298 4618 leader_election.cc:334] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bfa9d50481124d2e9f97ffe5a3ddc26f (127.28.47.190:42635): Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
I20260811 22:09:49.254475 5278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.253909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:42100)
0811 22:09:49.254162 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.254222 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":121,"threads_started":1}
W20260811 22:09:49.254606 4392 leader_election.cc:334] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer bfa9d50481124d2e9f97ffe5a3ddc26f (127.28.47.190:42635): Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
I20260811 22:09:49.256233 5270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.251327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:32861 (local address 127.0.0.1:58234)
0811 22:09:49.251633 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.251651 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:49.251789 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:49.252261 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:49.252265 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:49.252290 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:49.252401 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:49.252408 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.253441 (+ 1033us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.253446 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:49.254839 (+ 1393us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:49.254850 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.255890 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.255893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:49.255969 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:49.255992 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:09:49.256068 (+ 76us) client_negotiation.cc:240] Negotiation successful
0811 22:09:49.256129 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":59,"threads_started":1}
I20260811 22:09:49.256321 5272 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.251567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58234 (local address 127.28.47.189:32861)
0811 22:09:49.251862 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:49.251965 (+ 103us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:49.251982 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:09:49.252035 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:49.252037 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:49.252091 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:49.252177 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:49.252546 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.253310 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.254972 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.255740 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.255853 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:49.255916 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:49.256102 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 22:09:49.256176 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":62,"server-negotiator.queue_time_us":212,"thread_start_us":93,"threads_started":1}
I20260811 22:09:49.256807 4431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" is_pre_election: true
I20260811 22:09:49.257010 4431 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 70a6d0aaab4b4d6b8f6da676b5016064 in term 1.
I20260811 22:09:49.257357 4619 leader_election.cc:302] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 70a6d0aaab4b4d6b8f6da676b5016064, 83b6f0f1668c4e0b8e60b5d3d4ea8a60; no voters: bfa9d50481124d2e9f97ffe5a3ddc26f
I20260811 22:09:49.257520 5274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.253092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:35257 (local address 127.0.0.1:52450)
0811 22:09:49.253433 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.253446 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:49.253522 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:49.253875 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:49.253879 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:49.253896 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:49.253982 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:49.253990 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.254972 (+ 982us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.254976 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:49.256244 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:49.256253 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.257087 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.257091 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:49.257223 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:49.257248 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:09:49.257330 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 22:09:49.257409 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":273,"mutex_wait_us":72}
I20260811 22:09:49.257576 5275 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.253358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52450 (local address 127.28.47.188:35257)
0811 22:09:49.253664 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:49.253673 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:49.253687 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:09:49.253728 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:49.253731 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:49.253772 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:49.253869 (+ 97us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:49.254111 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.254856 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.256393 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.256896 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.257052 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:49.257145 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:49.257367 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 22:09:49.257431 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":118,"threads_started":1}
I20260811 22:09:49.257644 5266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260811 22:09:49.257710 5266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 1 FOLLOWER]: Starting leader election (detected failure of leader bfa9d50481124d2e9f97ffe5a3ddc26f)
I20260811 22:09:49.257774 5266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:09:49.257982 4649 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" is_pre_election: true
I20260811 22:09:49.258968 5266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } }
I20260811 22:09:49.259251 4649 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 83b6f0f1668c4e0b8e60b5d3d4ea8a60 in current term 2: Already voted for candidate 70a6d0aaab4b4d6b8f6da676b5016064 in this term.
I20260811 22:09:49.259411 5266 leader_election.cc:288] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [CANDIDATE]: Term 2 election: Requested vote from peers bfa9d50481124d2e9f97ffe5a3ddc26f (127.28.47.190:42635), 83b6f0f1668c4e0b8e60b5d3d4ea8a60 (127.28.47.189:32861)
I20260811 22:09:49.259894 5270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.259541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:42102)
0811 22:09:49.259763 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.259816 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151}
I20260811 22:09:49.259886 4431 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60"
I20260811 22:09:49.260007 4431 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:09:49.260007 4392 leader_election.cc:302] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 83b6f0f1668c4e0b8e60b5d3d4ea8a60; no voters: 70a6d0aaab4b4d6b8f6da676b5016064, bfa9d50481124d2e9f97ffe5a3ddc26f
I20260811 22:09:49.261214 5270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.260851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:42112)
0811 22:09:49.261010 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.261109 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
W20260811 22:09:49.261387 4618 leader_election.cc:334] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer bfa9d50481124d2e9f97ffe5a3ddc26f (127.28.47.190:42635): Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
I20260811 22:09:49.261662 4431 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 70a6d0aaab4b4d6b8f6da676b5016064 in term 2.
I20260811 22:09:49.261801 5267 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:09:49.262331 4619 leader_election.cc:302] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 70a6d0aaab4b4d6b8f6da676b5016064, 83b6f0f1668c4e0b8e60b5d3d4ea8a60; no voters: bfa9d50481124d2e9f97ffe5a3ddc26f
I20260811 22:09:49.262573 5266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:09:49.262851 5266 raft_consensus.cc:686] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [term 2 LEADER]: Becoming Leader. State: Replica: 70a6d0aaab4b4d6b8f6da676b5016064, State: Running, Role: LEADER
I20260811 22:09:49.263166 5266 consensus_queue.cc:237] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } }
I20260811 22:09:49.264637 5279 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70a6d0aaab4b4d6b8f6da676b5016064. Latest consensus state: current_term: 2 leader_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } } }
I20260811 22:09:49.264789 5279 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:49.265106 5282 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:49.267848 5282 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:49.268289 5282 catalog_manager.cc:1259] Loaded cluster ID: 700cadfaed6c4528aaef3158f64f6b98
I20260811 22:09:49.268362 5282 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:49.268947 5282 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:49.269409 5282 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: Loaded TSK: 0
I20260811 22:09:49.270160 5282 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:09:49.280476 4630 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:09:49.284047 4630 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:49.287833 5270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.287506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:42132)
0811 22:09:49.287657 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.287730 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 22:09:49.287981 4431 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [term 2 FOLLOWER]: Refusing update from remote peer 70a6d0aaab4b4d6b8f6da676b5016064: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260811 22:09:49.288623 5270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.288368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.0.0.1:42140)
0811 22:09:49.288479 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.288530 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260811 22:09:49.288539 5266 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [LEADER]: Connected to new peer: Peer: permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
W20260811 22:09:49.288784 4618 consensus_peers.cc:487] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 -> Peer bfa9d50481124d2e9f97ffe5a3ddc26f (127.28.47.190:42635): Couldn't send request to peer bfa9d50481124d2e9f97ffe5a3ddc26f. Status: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260811 22:09:49.290423 5267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 70a6d0aaab4b4d6b8f6da676b5016064. Latest consensus state: current_term: 2 leader_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } } }
I20260811 22:09:49.290581 5267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:09:49.291925 5266 sys_catalog.cc:455] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } } }
I20260811 22:09:49.292086 5266 sys_catalog.cc:458] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:49.292251 5267 sys_catalog.cc:455] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "bfa9d50481124d2e9f97ffe5a3ddc26f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42635 } } peers { permanent_uuid: "83b6f0f1668c4e0b8e60b5d3d4ea8a60" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 32861 } attrs { promote: false } } peers { permanent_uuid: "70a6d0aaab4b4d6b8f6da676b5016064" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 35257 } attrs { promote: false } } }
I20260811 22:09:49.292382 5267 sys_catalog.cc:458] T 00000000000000000000000000000000 P 83b6f0f1668c4e0b8e60b5d3d4ea8a60 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:09:49.303869 5270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.299411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:39521 (local address 127.0.0.1:45554)
0811 22:09:49.299620 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.299638 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:49.299729 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:49.300091 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:49.300094 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:49.300112 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:49.300213 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:49.300219 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.301275 (+ 1056us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.301279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:49.302418 (+ 1139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:49.302427 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.303395 (+ 968us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.303402 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:49.303529 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:49.303563 (+ 34us) client_negotiation.cc:769] Sending connection context
0811 22:09:49.303617 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 22:09:49.303712 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":127}
I20260811 22:09:49.304210 5269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.300036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:39211 (local address 127.0.0.1:40302)
0811 22:09:49.300182 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.300201 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:49.300305 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:49.300661 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:49.300667 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:49.300696 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:49.300816 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:49.300826 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.301904 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.301907 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:49.302840 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:49.302850 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.303840 (+ 990us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.303844 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:49.303933 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:49.303958 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:09:49.304011 (+ 53us) client_negotiation.cc:240] Negotiation successful
0811 22:09:49.304092 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260811 22:09:49.304574 5290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.299509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45554 (local address 127.28.47.129:39521)
0811 22:09:49.299814 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:49.299820 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:49.299833 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:09:49.299894 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:49.299896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:49.299973 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:49.300077 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:49.300361 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.301111 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.302588 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.303232 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.304241 (+ 1009us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:49.304314 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:49.304352 (+ 38us) server_negotiation.cc:299] Negotiation successful
0811 22:09:49.304441 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":113,"threads_started":1}
I20260811 22:09:49.304943 5291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.300137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40302 (local address 127.28.47.131:39211)
0811 22:09:49.300414 (+ 277us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:49.300419 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:49.300433 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:09:49.300481 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:49.300484 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:49.300530 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:49.300621 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:49.300990 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.301782 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.302973 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.303700 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.304600 (+ 900us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:49.304691 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:49.304734 (+ 43us) server_negotiation.cc:299] Negotiation successful
0811 22:09:49.304809 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":83,"threads_started":1}
I20260811 22:09:49.305188 4768 tablet_service.cc:1467] Processing CreateTablet for tablet b9846914ceca482c96542c836d5f9cab (DEFAULT_TABLE table=test-table [id=498dd64771e94b9792f16473d3ce5803]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:49.305460 5094 tablet_service.cc:1467] Processing CreateTablet for tablet b9846914ceca482c96542c836d5f9cab (DEFAULT_TABLE table=test-table [id=498dd64771e94b9792f16473d3ce5803]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:49.305545 4768 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9846914ceca482c96542c836d5f9cab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:49.305685 5094 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9846914ceca482c96542c836d5f9cab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:49.306982 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.300757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:37765 (local address 127.0.0.1:55680)
0811 22:09:49.302027 (+ 1270us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.302040 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:49.302149 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:49.302565 (+ 416us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:49.302570 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:49.302588 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:49.302686 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:49.302692 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.303713 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.303715 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:49.305322 (+ 1607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:49.305333 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.306331 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.306334 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:49.306467 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:49.306494 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:09:49.306567 (+ 73us) client_negotiation.cc:240] Negotiation successful
0811 22:09:49.306648 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1204,"thread_start_us":118,"threads_started":1}
I20260811 22:09:49.308066 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.301278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55680 (local address 127.28.47.130:37765)
0811 22:09:49.301524 (+ 246us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:49.301528 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:49.302230 (+ 702us) server_negotiation.cc:407] Connection header received
0811 22:09:49.302288 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:49.302291 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:49.302337 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:49.302410 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:49.302816 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.303590 (+ 774us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.305485 (+ 1895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.306098 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.306205 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:49.306267 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:49.307807 (+ 1540us) server_negotiation.cc:299] Negotiation successful
0811 22:09:49.307920 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":94,"threads_started":1}
I20260811 22:09:49.308632 4931 tablet_service.cc:1467] Processing CreateTablet for tablet b9846914ceca482c96542c836d5f9cab (DEFAULT_TABLE table=test-table [id=498dd64771e94b9792f16473d3ce5803]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:49.308730 5294 tablet_bootstrap.cc:492] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408: Bootstrap starting.
I20260811 22:09:49.308785 5297 tablet_bootstrap.cc:492] T b9846914ceca482c96542c836d5f9cab P 0f37525d62de4040b227093cc227011a: Bootstrap starting.
I20260811 22:09:49.308969 4931 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b9846914ceca482c96542c836d5f9cab. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:49.309918 5294 tablet_bootstrap.cc:654] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:49.309918 5297 tablet_bootstrap.cc:654] T b9846914ceca482c96542c836d5f9cab P 0f37525d62de4040b227093cc227011a: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:49.311479 5294 tablet_bootstrap.cc:492] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408: No bootstrap required, opened a new log
I20260811 22:09:49.311602 5294 ts_tablet_manager.cc:1397] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:49.311952 5297 tablet_bootstrap.cc:492] T b9846914ceca482c96542c836d5f9cab P 0f37525d62de4040b227093cc227011a: No bootstrap required, opened a new log
I20260811 22:09:49.312050 5297 ts_tablet_manager.cc:1397] T b9846914ceca482c96542c836d5f9cab P 0f37525d62de4040b227093cc227011a: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:49.312131 5294 raft_consensus.cc:348] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:49.312289 5294 raft_consensus.cc:374] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:49.312326 5294 raft_consensus.cc:729] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 87d7918cb98a4bdf8fb75f538a3ad408, State: Initialized, Role: FOLLOWER
I20260811 22:09:49.312517 5294 consensus_queue.cc:260] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:49.312572 5297 raft_consensus.cc:348] T b9846914ceca482c96542c836d5f9cab P 0f37525d62de4040b227093cc227011a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:49.312678 5297 raft_consensus.cc:374] T b9846914ceca482c96542c836d5f9cab P 0f37525d62de4040b227093cc227011a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:49.312721 5297 raft_consensus.cc:729] T b9846914ceca482c96542c836d5f9cab P 0f37525d62de4040b227093cc227011a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0f37525d62de4040b227093cc227011a, State: Initialized, Role: FOLLOWER
I20260811 22:09:49.312860 5297 consensus_queue.cc:260] T b9846914ceca482c96542c836d5f9cab P 0f37525d62de4040b227093cc227011a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:49.313005 5294 ts_tablet_manager.cc:1428] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:09:49.313015 5302 tablet_bootstrap.cc:492] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61: Bootstrap starting.
I20260811 22:09:49.313232 5297 ts_tablet_manager.cc:1428] T b9846914ceca482c96542c836d5f9cab P 0f37525d62de4040b227093cc227011a: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:09:49.314188 5305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.313688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.129:56195)
0811 22:09:49.313979 (+ 291us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.314053 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":142,"threads_started":1}
I20260811 22:09:49.316064 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.314955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.131:33309)
0811 22:09:49.315684 (+ 729us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.315923 (+ 239us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":658,"thread_start_us":106,"threads_started":1}
I20260811 22:09:49.317076 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.316769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.131:52079)
0811 22:09:49.316902 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.316994 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260811 22:09:49.317080 5177 heartbeater.cc:503] Master 127.28.47.188:35257 was elected leader, sending a full tablet report...
I20260811 22:09:49.317294 5305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.315161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.129:50549)
0811 22:09:49.317110 (+ 1949us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.317205 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1858}
I20260811 22:09:49.318298 5302 tablet_bootstrap.cc:654] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:49.319972 5302 tablet_bootstrap.cc:492] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61: No bootstrap required, opened a new log
I20260811 22:09:49.320109 5302 ts_tablet_manager.cc:1397] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 22:09:49.320576 5302 raft_consensus.cc:348] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:49.320712 5302 raft_consensus.cc:374] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:49.320770 5302 raft_consensus.cc:729] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f99e42e45a6e4a668e580c173dbd1d61, State: Initialized, Role: FOLLOWER
I20260811 22:09:49.320930 5302 consensus_queue.cc:260] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:49.321970 5302 ts_tablet_manager.cc:1428] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260811 22:09:49.322253 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.321764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.130:36547)
0811 22:09:49.322059 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.322137 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":137,"threads_started":1}
I20260811 22:09:49.323101 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.322788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.130:33783)
0811 22:09:49.322960 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.323026 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":102}
I20260811 22:09:49.323418 5012 heartbeater.cc:503] Master 127.28.47.188:35257 was elected leader, sending a full tablet report...
W20260811 22:09:49.327293 5180 tablet.cc:2369] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:09:49.512581 5015 tablet.cc:2369] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:09:49.551210 5308 raft_consensus.cc:482] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:49.551384 5308 raft_consensus.cc:504] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:49.552027 5308 leader_election.cc:288] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0f37525d62de4040b227093cc227011a (127.28.47.129:39521), 87d7918cb98a4bdf8fb75f538a3ad408 (127.28.47.131:39211)
I20260811 22:09:49.552634 4790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9846914ceca482c96542c836d5f9cab" candidate_uuid: "f99e42e45a6e4a668e580c173dbd1d61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f37525d62de4040b227093cc227011a" is_pre_election: true
I20260811 22:09:49.552704 5121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9846914ceca482c96542c836d5f9cab" candidate_uuid: "f99e42e45a6e4a668e580c173dbd1d61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" is_pre_election: true
I20260811 22:09:49.552913 4790 raft_consensus.cc:2454] T b9846914ceca482c96542c836d5f9cab P 0f37525d62de4040b227093cc227011a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99e42e45a6e4a668e580c173dbd1d61 in term 0.
I20260811 22:09:49.552927 5121 raft_consensus.cc:2454] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f99e42e45a6e4a668e580c173dbd1d61 in term 0.
I20260811 22:09:49.553443 4879 leader_election.cc:302] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0f37525d62de4040b227093cc227011a, 87d7918cb98a4bdf8fb75f538a3ad408, f99e42e45a6e4a668e580c173dbd1d61; no voters:
I20260811 22:09:49.553691 5308 raft_consensus.cc:2781] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:49.553771 5308 raft_consensus.cc:482] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:49.553833 5308 raft_consensus.cc:3037] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:49.555019 5308 raft_consensus.cc:504] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:49.555483 5308 leader_election.cc:288] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [CANDIDATE]: Term 1 election: Requested vote from peers 0f37525d62de4040b227093cc227011a (127.28.47.129:39521), 87d7918cb98a4bdf8fb75f538a3ad408 (127.28.47.131:39211)
I20260811 22:09:49.555927 4790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9846914ceca482c96542c836d5f9cab" candidate_uuid: "f99e42e45a6e4a668e580c173dbd1d61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0f37525d62de4040b227093cc227011a"
I20260811 22:09:49.556025 5121 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b9846914ceca482c96542c836d5f9cab" candidate_uuid: "f99e42e45a6e4a668e580c173dbd1d61" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "87d7918cb98a4bdf8fb75f538a3ad408"
I20260811 22:09:49.556085 4790 raft_consensus.cc:3037] T b9846914ceca482c96542c836d5f9cab P 0f37525d62de4040b227093cc227011a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:49.556198 5121 raft_consensus.cc:3037] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:49.557353 4790 raft_consensus.cc:2454] T b9846914ceca482c96542c836d5f9cab P 0f37525d62de4040b227093cc227011a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99e42e45a6e4a668e580c173dbd1d61 in term 1.
I20260811 22:09:49.557363 5121 raft_consensus.cc:2454] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f99e42e45a6e4a668e580c173dbd1d61 in term 1.
I20260811 22:09:49.557808 4877 leader_election.cc:302] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 87d7918cb98a4bdf8fb75f538a3ad408, f99e42e45a6e4a668e580c173dbd1d61; no voters:
I20260811 22:09:49.558068 5308 raft_consensus.cc:2781] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:49.558331 5308 raft_consensus.cc:686] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [term 1 LEADER]: Becoming Leader. State: Replica: f99e42e45a6e4a668e580c173dbd1d61, State: Running, Role: LEADER
I20260811 22:09:49.558526 5308 consensus_queue.cc:237] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } } peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } }
I20260811 22:09:49.559211 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.558942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.130:58601)
0811 22:09:49.559070 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.559139 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260811 22:09:49.560575 4627 catalog_manager.cc:5495] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 reported cstate change: term changed from 0 to 1, leader changed from <none> to f99e42e45a6e4a668e580c173dbd1d61 (127.28.47.130). New cstate: current_term: 1 leader_uuid: "f99e42e45a6e4a668e580c173dbd1d61" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f99e42e45a6e4a668e580c173dbd1d61" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37765 } health_report { overall_health: HEALTHY } } }
I20260811 22:09:49.605702 5293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.602126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55694 (local address 127.28.47.130:37765)
0811 22:09:49.602344 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:49.602350 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:49.602367 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:09:49.602428 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:49.602431 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:49.602474 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:49.602563 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:49.602885 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.603568 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.604500 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:49.604727 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:49.604842 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:49.605129 (+ 287us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:49.605276 (+ 147us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:49.605360 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:49.605487 (+ 127us) server_negotiation.cc:299] Negotiation successful
0811 22:09:49.605549 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:09:49.608353 4790 raft_consensus.cc:1261] T b9846914ceca482c96542c836d5f9cab P 0f37525d62de4040b227093cc227011a [term 1 FOLLOWER]: Refusing update from remote peer f99e42e45a6e4a668e580c173dbd1d61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:49.608374 5121 raft_consensus.cc:1261] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408 [term 1 FOLLOWER]: Refusing update from remote peer f99e42e45a6e4a668e580c173dbd1d61: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:49.608914 5308 consensus_queue.cc:1035] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0f37525d62de4040b227093cc227011a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39521 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:49.609144 5310 consensus_queue.cc:1035] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [LEADER]: Connected to new peer: Peer: permanent_uuid: "87d7918cb98a4bdf8fb75f538a3ad408" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 39211 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:49.610394 4848 heartbeater.cc:503] Master 127.28.47.188:35257 was elected leader, sending a full tablet report...
I20260811 22:09:49.611295 5305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.610968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.129:52683)
0811 22:09:49.611136 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.611198 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101}
I20260811 22:09:49.611369 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.611005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.131:38985)
0811 22:09:49.611195 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.611252 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260811 22:09:49.614570 5313 mvcc.cc:205] Tried to move back new op lower bound from 7317447432630636544 to 7317447432433528832. Current Snapshot: MvccSnapshot[applied={T|T < 7317447432630636544}]
I20260811 22:09:49.615742 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.615398 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.130:42757)
0811 22:09:49.615579 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.615649 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 22:09:49.616163 5306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.615595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.131:49487)
0811 22:09:49.616019 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.616083 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":347}
I20260811 22:09:49.616675 5309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.616319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.130:50007)
0811 22:09:49.616546 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.616601 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155}
I20260811 22:09:49.617429 5317 mvcc.cc:205] Tried to move back new op lower bound from 7317447432630636544 to 7317447432433528832. Current Snapshot: MvccSnapshot[applied={T|T < 7317447432630636544}]
I20260811 22:09:49.617815 5305 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.617460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42635 (local address 127.28.47.129:38399)
0811 22:09:49.617677 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.617730 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:42635: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":152}
I20260811 22:09:49.618633 5316 mvcc.cc:205] Tried to move back new op lower bound from 7317447432630636544 to 7317447432433528832. Current Snapshot: MvccSnapshot[applied={T|T < 7317447432630636544 or (T in {7317447432630636544})}]
I20260811 22:09:49.623558 4630 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:49.623739 4630 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:49.627985 4630 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 70a6d0aaab4b4d6b8f6da676b5016064: Sending DeleteTablet for 3 replicas of tablet b9846914ceca482c96542c836d5f9cab
I20260811 22:09:49.628960 4931 tablet_service.cc:1514] Processing DeleteTablet for tablet b9846914ceca482c96542c836d5f9cab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:49 UTC) from {username='oryx', principal='oryx/127.28.47.188@KRBTEST.COM'} at 127.0.0.1:55680
I20260811 22:09:49.629012 5094 tablet_service.cc:1514] Processing DeleteTablet for tablet b9846914ceca482c96542c836d5f9cab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:49 UTC) from {username='oryx', principal='oryx/127.28.47.188@KRBTEST.COM'} at 127.0.0.1:40302
I20260811 22:09:49.629025 4768 tablet_service.cc:1514] Processing DeleteTablet for tablet b9846914ceca482c96542c836d5f9cab with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:49 UTC) from {username='oryx', principal='oryx/127.28.47.188@KRBTEST.COM'} at 127.0.0.1:45554
I20260811 22:09:49.629463 5325 tablet_replica.cc:330] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408: stopping tablet replica
I20260811 22:09:49.629462 5324 tablet_replica.cc:330] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61: stopping tablet replica
I20260811 22:09:49.629606 5325 raft_consensus.cc:2229] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:49.629607 5324 raft_consensus.cc:2229] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:09:49.629739 5325 raft_consensus.cc:2258] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:49.629964 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 4589
I20260811 22:09:49.642467 5324 raft_consensus.cc:2258] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:49.642620 5325 ts_tablet_manager.cc:1905] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:49.643388 5324 ts_tablet_manager.cc:1905] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:49.645742 5324 ts_tablet_manager.cc:1918] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:49.645870 5324 log.cc:1196] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/b9846914ceca482c96542c836d5f9cab
I20260811 22:09:49.646152 5324 ts_tablet_manager.cc:1939] T b9846914ceca482c96542c836d5f9cab P f99e42e45a6e4a668e580c173dbd1d61: Deleting consensus metadata
I20260811 22:09:49.646214 5325 ts_tablet_manager.cc:1918] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:49.646327 5325 log.cc:1196] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/b9846914ceca482c96542c836d5f9cab
I20260811 22:09:49.646662 5325 ts_tablet_manager.cc:1939] T b9846914ceca482c96542c836d5f9cab P 87d7918cb98a4bdf8fb75f538a3ad408: Deleting consensus metadata
I20260811 22:09:49.647405 4621 catalog_manager.cc:4841] TS f99e42e45a6e4a668e580c173dbd1d61 (127.28.47.130:37765): tablet b9846914ceca482c96542c836d5f9cab (table test-table [id=498dd64771e94b9792f16473d3ce5803]) successfully deleted
I20260811 22:09:49.647691 4618 catalog_manager.cc:4841] TS 87d7918cb98a4bdf8fb75f538a3ad408 (127.28.47.131:39211): tablet b9846914ceca482c96542c836d5f9cab (table test-table [id=498dd64771e94b9792f16473d3ce5803]) successfully deleted
I20260811 22:09:49.650297 5292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:49.649966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:39521 (local address 127.0.0.1:45570)
0811 22:09:49.650141 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:49.650203 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:39521: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 22:09:49.650309 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 4857
W20260811 22:09:49.650525 4620 catalog_manager.cc:4584] TS 0f37525d62de4040b227093cc227011a (127.28.47.129:39521): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet b9846914ceca482c96542c836d5f9cab: Network error: Client connection negotiation failed: client connection to 127.28.47.129:39521: connect: Connection refused (error 111)
I20260811 22:09:49.662983 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 5021
I20260811 22:09:49.679812 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 4245
I20260811 22:09:49.690567 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 4369
2026-08-11T22:09:49Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6047 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:49 dist-test-slave-vxj2 krb5kdc[5331](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:49 dist-test-slave-vxj2 krb5kdc[5331](info): set up 2 sockets
Aug 11 22:09:49 dist-test-slave-vxj2 krb5kdc[5331](info): commencing operation
krb5kdc: starting...
W20260811 22:09:50.864424 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.144s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:09:50 dist-test-slave-vxj2 krb5kdc[5331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:50Z Disabled control of system clock
WARNING: no policy specified for oryx/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.190.keytab.
Entry for principal oryx/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.190.keytab.
I20260811 22:09:51.006642 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:38251
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.190.keytab
--principal=oryx/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:32837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:38251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:51.133744 5349 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:51.134153 5349 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:51.134243 5349 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:51.138724 5349 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:51.138864 5349 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260811 22:09:51.138898 5349 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:51.138947 5349 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:51.139005 5349 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:51.139048 5349 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:51.144968 5349 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:32837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:38251
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.190.keytab
--principal=oryx/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:38251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5349
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:51.146314 5349 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:51.147744 5349 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:51.157871 5354 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:51.158128 5357 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:51.158130 5349 server_base.cc:1034] running on GCE node
W20260811 22:09:51.157871 5355 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:51.159231 5349 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:51.160518 5349 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:51.161769 5349 hybrid_clock.cc:648] HybridClock initialized: now 1786486191161743 us; error 35 us; skew 500 ppm
Aug 11 22:09:51 dist-test-slave-vxj2 krb5kdc[5331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486191, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:51.169811 5349 init.cc:377] Logged in from keytab as oryx/127.28.47.190@KRBTEST.COM (short username oryx)
I20260811 22:09:51.170993 5349 webserver.cc:460] Webserver started at http://127.28.47.190:35165/ using document root <none> and password file <none>
I20260811 22:09:51.171617 5349 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:51.171684 5349 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:51.171886 5349 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:51.174263 5349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "f22772c039c340eeb24421c8af2519c9"
format_stamp: "Formatted at 2026-08-11 22:09:51 on dist-test-slave-vxj2"
server_key: "5fa50014269f8cbd269bab0f9222158a"
server_key_iv: "defbf0a57f4494e7418942799d39d3c8"
server_key_version: "encryptionkey@0"
I20260811 22:09:51.174865 5349 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "f22772c039c340eeb24421c8af2519c9"
format_stamp: "Formatted at 2026-08-11 22:09:51 on dist-test-slave-vxj2"
server_key: "5fa50014269f8cbd269bab0f9222158a"
server_key_iv: "defbf0a57f4494e7418942799d39d3c8"
server_key_version: "encryptionkey@0"
I20260811 22:09:51.179422 5349 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:09:51.182811 5365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:51.184814 5349 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:09:51.185009 5349 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "f22772c039c340eeb24421c8af2519c9"
format_stamp: "Formatted at 2026-08-11 22:09:51 on dist-test-slave-vxj2"
server_key: "5fa50014269f8cbd269bab0f9222158a"
server_key_iv: "defbf0a57f4494e7418942799d39d3c8"
server_key_version: "encryptionkey@0"
I20260811 22:09:51.185235 5349 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:51.200249 5349 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:51.204078 5349 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:51.204347 5349 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:51.214672 5349 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:38251
I20260811 22:09:51.214690 5446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:38251 every 8 connection(s)
I20260811 22:09:51.215885 5349 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:51.218982 5447 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:51.225459 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 5349
I20260811 22:09:51.225615 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:51.225912 28862 external_mini_cluster.cc:1428] Setting key 758f2a3e0cb5a6970cb18125b8083fa0
I20260811 22:09:51.225893 5447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9: Bootstrap starting.
I20260811 22:09:51.228765 5447 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:51.229655 5447 log.cc:824] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:51.231801 5447 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9: No bootstrap required, opened a new log
Aug 11 22:09:51 dist-test-slave-vxj2 krb5kdc[5331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486190, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:51.235672 5447 raft_consensus.cc:348] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22772c039c340eeb24421c8af2519c9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38251 } }
I20260811 22:09:51.236012 5447 raft_consensus.cc:374] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:51.236073 5447 raft_consensus.cc:729] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f22772c039c340eeb24421c8af2519c9, State: Initialized, Role: FOLLOWER
I20260811 22:09:51.236707 5447 consensus_queue.cc:260] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22772c039c340eeb24421c8af2519c9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38251 } }
I20260811 22:09:51.236912 5447 raft_consensus.cc:388] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:51.237000 5447 raft_consensus.cc:482] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:51.237115 5447 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:51.238304 5447 raft_consensus.cc:504] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22772c039c340eeb24421c8af2519c9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38251 } }
I20260811 22:09:51.238713 5447 leader_election.cc:302] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f22772c039c340eeb24421c8af2519c9; no voters:
I20260811 22:09:51.239069 5447 leader_election.cc:288] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:51.239236 5453 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:51.239516 5453 raft_consensus.cc:686] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [term 1 LEADER]: Becoming Leader. State: Replica: f22772c039c340eeb24421c8af2519c9, State: Running, Role: LEADER
I20260811 22:09:51.239894 5453 consensus_queue.cc:237] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22772c039c340eeb24421c8af2519c9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38251 } }
I20260811 22:09:51.240682 5447 sys_catalog.cc:564] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:51.242031 5455 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f22772c039c340eeb24421c8af2519c9. Latest consensus state: current_term: 1 leader_uuid: "f22772c039c340eeb24421c8af2519c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22772c039c340eeb24421c8af2519c9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38251 } } }
I20260811 22:09:51.242205 5455 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:51.242031 5454 sys_catalog.cc:455] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f22772c039c340eeb24421c8af2519c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f22772c039c340eeb24421c8af2519c9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38251 } } }
I20260811 22:09:51.242254 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:51.227415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58304 (local address 127.28.47.190:38251)
0811 22:09:51.227978 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:51.227996 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:51.228036 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:09:51.228737 (+ 701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:51.228756 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:51.229256 (+ 500us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:51.229627 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:51.230493 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.231685 (+ 1192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.232666 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.233239 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.233433 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:51.236175 (+ 2742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:51.236185 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:51.236229 (+ 44us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:51.236274 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:51.238422 (+ 2148us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:51.238945 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:51.238951 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:51.238957 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:51.239136 (+ 179us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:51.239580 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:51.239584 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:51.239586 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:51.240149 (+ 563us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:51.240361 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:51.240714 (+ 353us) server_negotiation.cc:299] Negotiation successful
0811 22:09:51.241071 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":380,"thread_start_us":159,"threads_started":1}
I20260811 22:09:51.242326 5454 sys_catalog.cc:458] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:51.242653 5464 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:51.246094 5464 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:51.252043 5464 catalog_manager.cc:1347] Generated new cluster ID: 82485b210d8f425fb1251915bda5c006
I20260811 22:09:51.252149 5464 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:51.287410 5464 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:51.288615 5464 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:51.299417 5464 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9: Generated new TSK 0
I20260811 22:09:51.300264 5464 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab.
Entry for principal oryx/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab.
I20260811 22:09:51.342976 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38251
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab
--principal=oryx/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:32837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:51.484102 5479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:51.484454 5479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:51.484503 5479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:51.489018 5479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:51.489112 5479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:51.489173 5479 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:09:51.489209 5479 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:51.495138 5479 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:32837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab
--principal=oryx/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.47.190:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:51.496439 5479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:51.497660 5479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:51.509070 5485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:51.509119 5479 server_base.cc:1034] running on GCE node
W20260811 22:09:51.509056 5486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:51.509056 5488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:51.509980 5479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:51.510789 5479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:51.512022 5479 hybrid_clock.cc:648] HybridClock initialized: now 1786486191511991 us; error 42 us; skew 500 ppm
Aug 11 22:09:51 dist-test-slave-vxj2 krb5kdc[5331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486191, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:51.515619 5479 init.cc:377] Logged in from keytab as oryx/127.28.47.129@KRBTEST.COM (short username oryx)
I20260811 22:09:51.516856 5479 webserver.cc:460] Webserver started at http://127.28.47.129:35681/ using document root <none> and password file <none>
I20260811 22:09:51.517524 5479 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:51.517596 5479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:51.517783 5479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:51.520015 5479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a"
format_stamp: "Formatted at 2026-08-11 22:09:51 on dist-test-slave-vxj2"
server_key: "46ac5f4d494a1c198dff8a726738e21e"
server_key_iv: "c7936405616a679dd132142c2ae45638"
server_key_version: "encryptionkey@0"
I20260811 22:09:51.520608 5479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a"
format_stamp: "Formatted at 2026-08-11 22:09:51 on dist-test-slave-vxj2"
server_key: "46ac5f4d494a1c198dff8a726738e21e"
server_key_iv: "c7936405616a679dd132142c2ae45638"
server_key_version: "encryptionkey@0"
I20260811 22:09:51.524940 5479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260811 22:09:51.527925 5498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:51.529525 5479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:51.529650 5479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a"
format_stamp: "Formatted at 2026-08-11 22:09:51 on dist-test-slave-vxj2"
server_key: "46ac5f4d494a1c198dff8a726738e21e"
server_key_iv: "c7936405616a679dd132142c2ae45638"
server_key_version: "encryptionkey@0"
I20260811 22:09:51.529829 5479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:51.553444 5479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:51.557101 5479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:51.557399 5479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:51.559118 5479 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:51.559206 5479 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:51.559289 5479 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:51.559333 5479 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:09:51 dist-test-slave-vxj2 krb5kdc[5331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486191, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:51.571604 5479 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:37385
I20260811 22:09:51.571820 5649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:37385 every 8 connection(s)
I20260811 22:09:51.572652 5479 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:51.574764 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:51.560043 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:44687 (local address 127.28.47.190:38251)
0811 22:09:51.560191 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:51.560194 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:51.561534 (+ 1340us) server_negotiation.cc:407] Connection header received
0811 22:09:51.562882 (+ 1348us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:51.562886 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:51.562957 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:51.563072 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:51.564750 (+ 1678us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.565376 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.566697 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.567052 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.567201 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:51.570607 (+ 3406us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:51.570611 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:51.570613 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:51.570643 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:51.572267 (+ 1624us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:51.572890 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:51.572894 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:51.572896 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:51.573010 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:51.573488 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:51.573492 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:51.573493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:51.573750 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:51.573879 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:51.574423 (+ 544us) server_negotiation.cc:299] Negotiation successful
0811 22:09:51.574579 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:09:51.575673 5507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:51.560655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38251 (local address 127.28.47.129:44687)
0811 22:09:51.561339 (+ 684us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:51.561391 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:51.562608 (+ 1217us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:51.563272 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:51.563279 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:51.563803 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:51.564550 (+ 747us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:51.564569 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.565547 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.565551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:51.566548 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:51.566562 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.567315 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.567322 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:51.567614 (+ 292us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:51.568391 (+ 777us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:51.568414 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:51.570392 (+ 1978us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:51.572405 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:51.572411 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:51.572427 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:51.572769 (+ 342us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:51.573138 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:51.573141 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:51.573143 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:51.573354 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:51.573877 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:51.573885 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:51.574155 (+ 270us) client_negotiation.cc:769] Sending connection context
0811 22:09:51.574357 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 22:09:51.574728 (+ 371us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":173,"threads_started":1}
I20260811 22:09:51.577029 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 5479
I20260811 22:09:51.577234 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:09:51.577255 5651 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38251
I20260811 22:09:51.577457 5651 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:51.577638 28862 external_mini_cluster.cc:1428] Setting key 6c86756763603633a7d5a0584d12c834
I20260811 22:09:51.578157 5651 heartbeater.cc:511] Master 127.28.47.190:38251 requested a full tablet report, sending...
I20260811 22:09:51.579964 5384 ts_manager.cc:194] Registered new tserver with Master: d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129:37385)
I20260811 22:09:51.581774 5384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:44687
I20260811 22:09:51.588234 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:51.579952 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:55293 (local address 127.28.47.190:38251)
0811 22:09:51.580093 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:51.580098 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:51.580536 (+ 438us) server_negotiation.cc:407] Connection header received
0811 22:09:51.580636 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:51.580642 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:51.580704 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:51.580797 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:51.581396 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.582173 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.583278 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.583503 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.583637 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:51.584903 (+ 1266us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:51.584907 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:51.584910 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:51.584954 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:51.586477 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:51.586935 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:51.586938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:51.586940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:51.587034 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:51.587332 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:51.587336 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:51.587337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:51.587537 (+ 200us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:51.587655 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:51.587942 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 22:09:51.588064 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:09:51.588243 5657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:51.579966 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38251 (local address 127.28.47.129:55293)
0811 22:09:51.580415 (+ 449us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:51.580432 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:51.580548 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:51.580838 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:51.580841 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:51.581096 (+ 255us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:51.581268 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:51.581276 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.582313 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.582316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:51.583114 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:51.583126 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.583646 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.583650 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:51.583753 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:51.584336 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:51.584356 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:51.584744 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:51.586599 (+ 1855us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:51.586604 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:51.586606 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:51.586821 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:51.587134 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:51.587138 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:51.587140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:51.587231 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:51.587642 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:51.587645 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:51.587760 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 22:09:51.587874 (+ 114us) client_negotiation.cc:240] Negotiation successful
0811 22:09:51.588035 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":357,"thread_start_us":239,"threads_started":1}
WARNING: no policy specified for oryx/127.28.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab.
Entry for principal oryx/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab.
I20260811 22:09:51.613462 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38251
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab
--principal=oryx/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:32837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:51.742208 5660 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:51.742686 5660 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:51.742779 5660 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:51.747326 5660 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:51.747407 5660 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:51.747486 5660 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:09:51.747558 5660 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:51.754055 5660 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:32837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab
--principal=oryx/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.47.190:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5660
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:51.755354 5660 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:51.756582 5660 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:51.767148 5670 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:51.767148 5667 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:51.767189 5668 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:51.767477 5660 server_base.cc:1034] running on GCE node
I20260811 22:09:51.768020 5660 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:51.768730 5660 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:51.769942 5660 hybrid_clock.cc:648] HybridClock initialized: now 1786486191769860 us; error 46 us; skew 500 ppm
Aug 11 22:09:51 dist-test-slave-vxj2 krb5kdc[5331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486191, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:51.773039 5660 init.cc:377] Logged in from keytab as oryx/127.28.47.130@KRBTEST.COM (short username oryx)
I20260811 22:09:51.774089 5660 webserver.cc:460] Webserver started at http://127.28.47.130:45911/ using document root <none> and password file <none>
I20260811 22:09:51.774711 5660 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:51.774811 5660 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:51.775060 5660 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:51.777495 5660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "09707546d2e64cddacb610a7fcd85e6b"
format_stamp: "Formatted at 2026-08-11 22:09:51 on dist-test-slave-vxj2"
server_key: "5d3885728ea6454d64ac1c015c19ffa1"
server_key_iv: "14ea639865c386d4ac22e2128c0457aa"
server_key_version: "encryptionkey@0"
I20260811 22:09:51.778142 5660 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "09707546d2e64cddacb610a7fcd85e6b"
format_stamp: "Formatted at 2026-08-11 22:09:51 on dist-test-slave-vxj2"
server_key: "5d3885728ea6454d64ac1c015c19ffa1"
server_key_iv: "14ea639865c386d4ac22e2128c0457aa"
server_key_version: "encryptionkey@0"
I20260811 22:09:51.782423 5660 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:09:51.785102 5680 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:51.786517 5660 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:51.786674 5660 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "09707546d2e64cddacb610a7fcd85e6b"
format_stamp: "Formatted at 2026-08-11 22:09:51 on dist-test-slave-vxj2"
server_key: "5d3885728ea6454d64ac1c015c19ffa1"
server_key_iv: "14ea639865c386d4ac22e2128c0457aa"
server_key_version: "encryptionkey@0"
I20260811 22:09:51.786808 5660 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:51.810205 5660 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:51.814333 5660 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:51.814622 5660 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:51.816435 5660 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:51.816545 5660 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:51.816627 5660 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:51.816679 5660 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:09:51 dist-test-slave-vxj2 krb5kdc[5331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486191, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:51.829324 5660 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:38063
I20260811 22:09:51.829457 5816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:38063 every 8 connection(s)
I20260811 22:09:51.830377 5660 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:51.832059 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 5660
I20260811 22:09:51.832221 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:09:51.832590 28862 external_mini_cluster.cc:1428] Setting key 7712af58a48c6f674e86362b7633d58b
I20260811 22:09:51.834429 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:51.817698 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:33201 (local address 127.28.47.190:38251)
0811 22:09:51.817847 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:51.817851 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:51.819460 (+ 1609us) server_negotiation.cc:407] Connection header received
0811 22:09:51.821049 (+ 1589us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:51.821053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:51.821124 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:51.821297 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:51.822803 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.823565 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.825665 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.826002 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.826106 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:51.829886 (+ 3780us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:51.829890 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:51.829893 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:51.829932 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:51.831614 (+ 1682us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:51.832350 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:51.832358 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:51.832360 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:51.832510 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:51.832946 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:51.832949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:51.832951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:51.833259 (+ 308us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:51.833371 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:51.834070 (+ 699us) server_negotiation.cc:299] Negotiation successful
0811 22:09:51.834270 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:09:51.835611 5692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:51.818595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38251 (local address 127.28.47.130:33201)
0811 22:09:51.819217 (+ 622us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:51.819267 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:51.820791 (+ 1524us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:51.821508 (+ 717us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:51.821516 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:51.822071 (+ 555us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:51.822633 (+ 562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:51.822648 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.823734 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.823738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:51.825490 (+ 1752us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:51.825507 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.826203 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.826210 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:51.826484 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:51.827333 (+ 849us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:51.827361 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:51.829651 (+ 2290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:51.831790 (+ 2139us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:51.831796 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:51.831812 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:51.832204 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:51.832644 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:51.832647 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:51.832649 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:51.832837 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:51.833406 (+ 569us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:51.833415 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:51.833800 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 22:09:51.833999 (+ 199us) client_negotiation.cc:240] Negotiation successful
0811 22:09:51.834455 (+ 456us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":385,"thread_start_us":179,"threads_started":1}
I20260811 22:09:51.836994 5817 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38251
I20260811 22:09:51.837210 5817 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:51.837850 5817 heartbeater.cc:511] Master 127.28.47.190:38251 requested a full tablet report, sending...
I20260811 22:09:51.839167 5384 ts_manager.cc:194] Registered new tserver with Master: 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063)
I20260811 22:09:51.840021 5384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:33201
WARNING: no policy specified for oryx/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:09:51.848938 5825 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:51.839709 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38251 (local address 127.28.47.130:59479)
0811 22:09:51.840184 (+ 475us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:51.840203 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:51.840327 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:51.840677 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:51.840682 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:51.840949 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:51.841221 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:51.841235 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.842336 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.842343 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:51.843377 (+ 1034us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:51.843392 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.843967 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.843972 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:51.844096 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:51.844679 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:51.844694 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:51.845226 (+ 532us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:51.847158 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:51.847168 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:51.847169 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:51.847413 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:51.847715 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:51.847718 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:51.847719 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:51.847806 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:51.848337 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:51.848343 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:51.848462 (+ 119us) client_negotiation.cc:769] Sending connection context
0811 22:09:51.848555 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 22:09:51.848694 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":360,"mutex_wait_us":1,"thread_start_us":129,"threads_started":1}
I20260811 22:09:51.849004 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:51.839818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:59479 (local address 127.28.47.190:38251)
0811 22:09:51.839945 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:51.839950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:51.840271 (+ 321us) server_negotiation.cc:407] Connection header received
0811 22:09:51.840438 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:51.840441 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:51.840496 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:51.840588 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:51.841395 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.842161 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.843526 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:51.843822 (+ 296us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:51.843923 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:51.845375 (+ 1452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:51.845378 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:51.845380 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:51.845406 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:51.847032 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:51.847534 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:51.847537 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:51.847539 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:51.847625 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:51.847961 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:51.847964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:51.847965 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:51.848224 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:51.848323 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:51.848662 (+ 339us) server_negotiation.cc:299] Negotiation successful
0811 22:09:51.848795 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab.
Entry for principal oryx/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab.
I20260811 22:09:51.870515 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38251
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab
--principal=oryx/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:32837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:51.995575 5827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:51.995944 5827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:51.996029 5827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:52.000262 5827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:52.000381 5827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:52.000463 5827 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260811 22:09:52.000555 5827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:52.006322 5827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:32837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab
--principal=oryx/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.28.47.190:38251
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:52.007592 5827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:52.008733 5827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:52.019574 5835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:52.019552 5839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:52.019667 5827 server_base.cc:1034] running on GCE node
W20260811 22:09:52.019552 5836 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:52.020388 5827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:52.021216 5827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:52.022431 5827 hybrid_clock.cc:648] HybridClock initialized: now 1786486192022402 us; error 42 us; skew 500 ppm
Aug 11 22:09:52 dist-test-slave-vxj2 krb5kdc[5331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:52.025811 5827 init.cc:377] Logged in from keytab as oryx/127.28.47.131@KRBTEST.COM (short username oryx)
I20260811 22:09:52.026887 5827 webserver.cc:460] Webserver started at http://127.28.47.131:36989/ using document root <none> and password file <none>
I20260811 22:09:52.027630 5827 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:52.027732 5827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:52.027981 5827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:52.030339 5827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "fb28c8c5cf384328b2b6ced6b59ac57f"
format_stamp: "Formatted at 2026-08-11 22:09:52 on dist-test-slave-vxj2"
server_key: "14adfddfbf8dae5a90d190adbd61afaf"
server_key_iv: "47140d7992242ce55c3b30931affc6c1"
server_key_version: "encryptionkey@0"
I20260811 22:09:52.030977 5827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "fb28c8c5cf384328b2b6ced6b59ac57f"
format_stamp: "Formatted at 2026-08-11 22:09:52 on dist-test-slave-vxj2"
server_key: "14adfddfbf8dae5a90d190adbd61afaf"
server_key_iv: "47140d7992242ce55c3b30931affc6c1"
server_key_version: "encryptionkey@0"
I20260811 22:09:52.035337 5827 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:09:52.038206 5846 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:52.039619 5827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:52.039773 5827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "fb28c8c5cf384328b2b6ced6b59ac57f"
format_stamp: "Formatted at 2026-08-11 22:09:52 on dist-test-slave-vxj2"
server_key: "14adfddfbf8dae5a90d190adbd61afaf"
server_key_iv: "47140d7992242ce55c3b30931affc6c1"
server_key_version: "encryptionkey@0"
I20260811 22:09:52.039911 5827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:52.055753 5827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:52.059513 5827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:52.059795 5827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:52.061681 5827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:52.061779 5827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:52.061859 5827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:52.061924 5827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 11 22:09:52 dist-test-slave-vxj2 krb5kdc[5331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486192, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:52.074790 5827 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:38313
I20260811 22:09:52.075232 5983 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:38313 every 8 connection(s)
I20260811 22:09:52.075935 5827 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:52.078732 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.062491 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:48783 (local address 127.28.47.190:38251)
0811 22:09:52.062681 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.062686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.063653 (+ 967us) server_negotiation.cc:407] Connection header received
0811 22:09:52.065058 (+ 1405us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.065062 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.065123 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.065249 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:52.066723 (+ 1474us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.067269 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.069451 (+ 2182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.069820 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.069985 (+ 165us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.074010 (+ 4025us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:52.074016 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:52.074022 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:52.074066 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:52.075934 (+ 1868us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:52.076709 (+ 775us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:52.076714 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:52.076716 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:52.076854 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:52.077393 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:52.077397 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:52.077398 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:52.077660 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:52.077824 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.078441 (+ 617us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.078588 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:09:52.079524 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 5827
I20260811 22:09:52.079641 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:09:52.079713 5857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.062907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38251 (local address 127.28.47.131:48783)
0811 22:09:52.063446 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:52.063514 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:52.064784 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:52.065536 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:52.065542 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:52.066012 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:52.066546 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.066561 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.067472 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.067478 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.069139 (+ 1661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.069190 (+ 51us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.070043 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.070051 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.070324 (+ 273us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.071181 (+ 857us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:52.071203 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:52.073729 (+ 2526us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:52.076119 (+ 2390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:52.076126 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:52.076143 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:52.076555 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:52.076990 (+ 435us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:52.076996 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:52.076998 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:52.077240 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:52.077830 (+ 590us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:52.077835 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:52.078146 (+ 311us) client_negotiation.cc:769] Sending connection context
0811 22:09:52.078319 (+ 173us) client_negotiation.cc:240] Negotiation successful
0811 22:09:52.078607 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":102,"threads_started":1}
I20260811 22:09:52.080147 28862 external_mini_cluster.cc:1428] Setting key 3e87d7f595a78470bafbba87974b8585
I20260811 22:09:52.084951 5984 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38251
I20260811 22:09:52.085180 5984 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:52.085785 5984 heartbeater.cc:511] Master 127.28.47.190:38251 requested a full tablet report, sending...
I20260811 22:09:52.086935 5384 ts_manager.cc:194] Registered new tserver with Master: fb28c8c5cf384328b2b6ced6b59ac57f (127.28.47.131:38313)
I20260811 22:09:52.087720 5384 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:48783
I20260811 22:09:52.094240 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.085261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:54713 (local address 127.28.47.190:38251)
0811 22:09:52.085395 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.085399 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.085705 (+ 306us) server_negotiation.cc:407] Connection header received
0811 22:09:52.085822 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.085825 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.085901 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.086011 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:52.086570 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.087202 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.088315 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.088809 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.088936 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.090423 (+ 1487us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:52.090427 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:52.090430 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:52.090461 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:52.092120 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:52.092623 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:52.092628 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:52.092630 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:52.092723 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:52.093034 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:52.093039 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:52.093041 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:52.093485 (+ 444us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:52.093625 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.093933 (+ 308us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.094060 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260811 22:09:52.094256 5991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.085131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38251 (local address 127.28.47.131:54713)
0811 22:09:52.085610 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:52.085627 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:52.085734 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:52.086033 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:52.086036 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:52.086261 (+ 225us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:52.086418 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.086430 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.087353 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.087356 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.088172 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.088182 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.088978 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.088982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.089062 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.089825 (+ 763us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:52.089842 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:52.090219 (+ 377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:52.092262 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:52.092266 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:52.092268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:52.092503 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:52.092827 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:52.092830 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:52.092832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:52.092931 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:52.093612 (+ 681us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:52.093615 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:52.093732 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 22:09:52.093861 (+ 129us) client_negotiation.cc:240] Negotiation successful
0811 22:09:52.094063 (+ 202us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"thread_start_us":120,"threads_started":1}
I20260811 22:09:52.096556 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 11 22:09:52 dist-test-slave-vxj2 krb5kdc[5331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486192, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 11 22:09:52 dist-test-slave-vxj2 krb5kdc[5331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:52 dist-test-slave-vxj2 krb5kdc[5331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:09:52.124658 5451 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:52.117547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58306 (local address 127.28.47.190:38251)
0811 22:09:52.117726 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.117729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.117746 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:09:52.117791 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.117793 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.117835 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.117965 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:52.118779 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.119334 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.120257 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.120482 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.120600 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.124439 (+ 3839us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58306: BlockingRecv error: recv got EOF from 127.0.0.1:58306 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
Aug 11 22:09:52 dist-test-slave-vxj2 krb5kdc[5331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486192, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:52.136166 5451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.126549 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58310 (local address 127.28.47.190:38251)
0811 22:09:52.126738 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.126743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.126912 (+ 169us) server_negotiation.cc:407] Connection header received
0811 22:09:52.127090 (+ 178us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.127098 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.127142 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.127237 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:52.127922 (+ 685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.128453 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.129342 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.129568 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.129691 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.132425 (+ 2734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:52.132428 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:52.132430 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:52.132464 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:52.134184 (+ 1720us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:52.134739 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:52.134742 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:52.134744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:52.134845 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:52.135191 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:52.135194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:52.135196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:52.135428 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:52.135598 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.135840 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.135982 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90}
I20260811 22:09:52.140080 5383 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:09:52.143702 5383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:52.159888 6011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.154109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:38063 (local address 127.0.0.1:38544)
0811 22:09:52.154655 (+ 546us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:52.154670 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:52.154779 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:52.155733 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:52.155746 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:52.155772 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:52.155927 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.155935 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.157101 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.157107 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.158416 (+ 1309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.158426 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.159311 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.159314 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.159429 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.159475 (+ 46us) client_negotiation.cc:769] Sending connection context
0811 22:09:52.159595 (+ 120us) client_negotiation.cc:240] Negotiation successful
0811 22:09:52.159719 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"thread_start_us":113,"threads_started":1}
I20260811 22:09:52.159899 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.154682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38544 (local address 127.28.47.130:38063)
0811 22:09:52.155200 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.155213 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.155241 (+ 28us) server_negotiation.cc:407] Connection header received
0811 22:09:52.155327 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.155333 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.155558 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.155745 (+ 187us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:52.156050 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.156981 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.158549 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.159177 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.159291 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.159485 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.159636 (+ 151us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.159743 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":131,"threads_started":1}
I20260811 22:09:52.160687 6012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.154621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:38313 (local address 127.0.0.1:36180)
0811 22:09:52.155214 (+ 593us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:52.155231 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:52.155330 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:52.156235 (+ 905us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:52.156238 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:52.156257 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:52.156422 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.156430 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.157978 (+ 1548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.157982 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.159086 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.159097 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.160240 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.160244 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.160351 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.160379 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:09:52.160433 (+ 54us) client_negotiation.cc:240] Negotiation successful
0811 22:09:52.160511 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":100,"threads_started":1}
I20260811 22:09:52.161108 6010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.154023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37128 (local address 127.28.47.129:37385)
0811 22:09:52.155591 (+ 1568us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.155607 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.155634 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:09:52.155715 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.155722 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.156045 (+ 323us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.156218 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:52.156460 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.157656 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.159196 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.160005 (+ 809us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.160115 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.160291 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.160649 (+ 358us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.160978 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1450,"thread_start_us":100,"threads_started":1}
I20260811 22:09:52.160846 6009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.153880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:37385 (local address 127.0.0.1:37128)
0811 22:09:52.154259 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:52.154317 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:52.154445 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:52.156199 (+ 1754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:52.156202 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:52.156221 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:52.156329 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.156335 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.157805 (+ 1470us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.157809 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.159044 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.159056 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.160186 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.160190 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.160296 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.160471 (+ 175us) client_negotiation.cc:769] Sending connection context
0811 22:09:52.160593 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 22:09:52.160701 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"thread_start_us":95,"threads_started":1}
I20260811 22:09:52.161531 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.155148 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36180 (local address 127.28.47.131:38313)
0811 22:09:52.155696 (+ 548us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.155707 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.155727 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:09:52.155792 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.155796 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.156072 (+ 276us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.156600 (+ 528us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:52.156742 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.157860 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.159285 (+ 1425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.160101 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.160987 (+ 886us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.161192 (+ 205us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.161258 (+ 66us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.161381 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":447,"thread_start_us":132,"threads_started":1}
I20260811 22:09:52.164423 5565 tablet_service.cc:1467] Processing CreateTablet for tablet 03ff866918004986b5afee64b56739c7 (DEFAULT_TABLE table=test-table [id=d92a9f9ee0a64849b0c252cd72a34a03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:52.164464 5906 tablet_service.cc:1467] Processing CreateTablet for tablet 03ff866918004986b5afee64b56739c7 (DEFAULT_TABLE table=test-table [id=d92a9f9ee0a64849b0c252cd72a34a03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:52.164435 5739 tablet_service.cc:1467] Processing CreateTablet for tablet 03ff866918004986b5afee64b56739c7 (DEFAULT_TABLE table=test-table [id=d92a9f9ee0a64849b0c252cd72a34a03]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:52.165700 5906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03ff866918004986b5afee64b56739c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:52.165700 5739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03ff866918004986b5afee64b56739c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:52.165874 5565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 03ff866918004986b5afee64b56739c7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:52.172695 6015 tablet_bootstrap.cc:492] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a: Bootstrap starting.
I20260811 22:09:52.174685 6017 tablet_bootstrap.cc:492] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f: Bootstrap starting.
I20260811 22:09:52.175356 6015 tablet_bootstrap.cc:654] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:52.176163 6015 log.cc:824] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:52.176819 6017 tablet_bootstrap.cc:654] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:52.176970 6016 tablet_bootstrap.cc:492] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b: Bootstrap starting.
I20260811 22:09:52.177850 6017 log.cc:824] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:52.178808 6015 tablet_bootstrap.cc:492] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a: No bootstrap required, opened a new log
I20260811 22:09:52.179057 6015 ts_tablet_manager.cc:1397] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:09:52.179701 6016 tablet_bootstrap.cc:654] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:52.180137 6017 tablet_bootstrap.cc:492] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f: No bootstrap required, opened a new log
I20260811 22:09:52.180367 6017 ts_tablet_manager.cc:1397] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:09:52.180534 6016 log.cc:824] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:52.182650 6016 tablet_bootstrap.cc:492] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b: No bootstrap required, opened a new log
I20260811 22:09:52.182883 6016 ts_tablet_manager.cc:1397] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:09:52.184630 6015 raft_consensus.cc:348] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } }
I20260811 22:09:52.184994 6015 raft_consensus.cc:374] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:52.184839 6017 raft_consensus.cc:348] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } }
I20260811 22:09:52.185066 6017 raft_consensus.cc:374] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:52.185058 6015 raft_consensus.cc:729] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1adb8f1e4ae465ba20fe766bdafdd1a, State: Initialized, Role: FOLLOWER
I20260811 22:09:52.185113 6017 raft_consensus.cc:729] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb28c8c5cf384328b2b6ced6b59ac57f, State: Initialized, Role: FOLLOWER
I20260811 22:09:52.185662 6015 consensus_queue.cc:260] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } }
I20260811 22:09:52.185660 6017 consensus_queue.cc:260] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } }
I20260811 22:09:52.186863 6017 ts_tablet_manager.cc:1428] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260811 22:09:52.186863 6015 ts_tablet_manager.cc:1428] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a: Time spent starting tablet: real 0.008s user 0.004s sys 0.004s
I20260811 22:09:52.187285 5651 heartbeater.cc:503] Master 127.28.47.190:38251 was elected leader, sending a full tablet report...
I20260811 22:09:52.187964 6016 raft_consensus.cc:348] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } }
I20260811 22:09:52.188261 6016 raft_consensus.cc:374] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:52.188330 6016 raft_consensus.cc:729] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09707546d2e64cddacb610a7fcd85e6b, State: Initialized, Role: FOLLOWER
I20260811 22:09:52.188756 5984 heartbeater.cc:503] Master 127.28.47.190:38251 was elected leader, sending a full tablet report...
I20260811 22:09:52.188853 6016 consensus_queue.cc:260] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } }
I20260811 22:09:52.190114 6016 ts_tablet_manager.cc:1428] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:09:52.190541 5817 heartbeater.cc:503] Master 127.28.47.190:38251 was elected leader, sending a full tablet report...
W20260811 22:09:52.324393 5652 tablet.cc:2369] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:09:52.327729 5985 tablet.cc:2369] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:09:52.332644 5818 tablet.cc:2369] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:09:52.520100 6021 raft_consensus.cc:482] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:52.520428 6021 raft_consensus.cc:504] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } }
I20260811 22:09:52.522029 6021 leader_election.cc:288] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fb28c8c5cf384328b2b6ced6b59ac57f (127.28.47.131:38313), 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063)
I20260811 22:09:52.528069 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.522454 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:52185 (local address 127.28.47.131:38313)
0811 22:09:52.522588 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.522593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.522870 (+ 277us) server_negotiation.cc:407] Connection header received
0811 22:09:52.523045 (+ 175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.523048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.523104 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.523199 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:52.523752 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.524636 (+ 884us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.526618 (+ 1982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.527259 (+ 641us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.527410 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.527471 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.527833 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.527949 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:09:52.528069 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.522450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:53453 (local address 127.28.47.130:38063)
0811 22:09:52.522588 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.522593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.522938 (+ 345us) server_negotiation.cc:407] Connection header received
0811 22:09:52.523045 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.523048 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.523104 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.523199 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:52.523752 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.524631 (+ 879us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.526617 (+ 1986us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.527259 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.527411 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.527471 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.527833 (+ 362us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.527930 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:09:52.528112 6027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.522277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:38313 (local address 127.28.47.129:52185)
0811 22:09:52.522754 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:52.522771 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:52.522921 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:52.523224 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:52.523228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:52.523274 (+ 46us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:52.523567 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.523574 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.524906 (+ 1332us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.524909 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.526429 (+ 1520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.526436 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.527451 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.527455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.527631 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.527681 (+ 50us) client_negotiation.cc:769] Sending connection context
0811 22:09:52.527777 (+ 96us) client_negotiation.cc:240] Negotiation successful
0811 22:09:52.527880 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":377,"mutex_wait_us":268,"spinlock_wait_cycles":39168,"thread_start_us":145,"threads_started":1}
I20260811 22:09:52.528112 6026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.522277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:38063 (local address 127.28.47.129:53453)
0811 22:09:52.522847 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:52.522860 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:52.522938 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:52.523224 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:52.523228 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:52.523275 (+ 47us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:52.523566 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.523574 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.524856 (+ 1282us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.524860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.526422 (+ 1562us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.526436 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.527451 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.527455 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.527625 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.527708 (+ 83us) client_negotiation.cc:769] Sending connection context
0811 22:09:52.527779 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:09:52.527880 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":495,"mutex_wait_us":250,"spinlock_wait_cycles":8960,"thread_start_us":104,"threads_started":1}
I20260811 22:09:52.529016 5762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03ff866918004986b5afee64b56739c7" candidate_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09707546d2e64cddacb610a7fcd85e6b" is_pre_election: true
I20260811 22:09:52.529021 5930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03ff866918004986b5afee64b56739c7" candidate_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" is_pre_election: true
I20260811 22:09:52.529438 5762 raft_consensus.cc:2454] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1adb8f1e4ae465ba20fe766bdafdd1a in term 0.
I20260811 22:09:52.529444 5930 raft_consensus.cc:2454] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1adb8f1e4ae465ba20fe766bdafdd1a in term 0.
I20260811 22:09:52.530040 5499 leader_election.cc:302] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 09707546d2e64cddacb610a7fcd85e6b, d1adb8f1e4ae465ba20fe766bdafdd1a, fb28c8c5cf384328b2b6ced6b59ac57f; no voters:
I20260811 22:09:52.530372 6021 raft_consensus.cc:2781] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:52.530514 6021 raft_consensus.cc:482] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:52.530601 6021 raft_consensus.cc:3037] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:52.532063 6021 raft_consensus.cc:504] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } }
I20260811 22:09:52.532519 6021 leader_election.cc:288] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [CANDIDATE]: Term 1 election: Requested vote from peers fb28c8c5cf384328b2b6ced6b59ac57f (127.28.47.131:38313), 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063)
I20260811 22:09:52.532995 5930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03ff866918004986b5afee64b56739c7" candidate_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f"
I20260811 22:09:52.533154 5762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "03ff866918004986b5afee64b56739c7" candidate_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09707546d2e64cddacb610a7fcd85e6b"
I20260811 22:09:52.533159 5930 raft_consensus.cc:3037] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:52.533329 5762 raft_consensus.cc:3037] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:52.534631 5762 raft_consensus.cc:2454] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1adb8f1e4ae465ba20fe766bdafdd1a in term 1.
I20260811 22:09:52.535122 5501 leader_election.cc:302] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09707546d2e64cddacb610a7fcd85e6b, d1adb8f1e4ae465ba20fe766bdafdd1a; no voters:
I20260811 22:09:52.535305 5930 raft_consensus.cc:2454] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1adb8f1e4ae465ba20fe766bdafdd1a in term 1.
I20260811 22:09:52.535415 6021 raft_consensus.cc:2781] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:52.535693 6021 raft_consensus.cc:686] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 LEADER]: Becoming Leader. State: Replica: d1adb8f1e4ae465ba20fe766bdafdd1a, State: Running, Role: LEADER
I20260811 22:09:52.536134 6021 consensus_queue.cc:237] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } }
I20260811 22:09:52.539654 5384 catalog_manager.cc:5495] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a reported cstate change: term changed from 0 to 1, leader changed from <none> to d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129). New cstate: current_term: 1 leader_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } health_report { overall_health: UNKNOWN } } }
I20260811 22:09:52.620754 6010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.614567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37132 (local address 127.28.47.129:37385)
0811 22:09:52.614781 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.614789 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.614811 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:09:52.615020 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.615025 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.615122 (+ 97us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.615274 (+ 152us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:52.615548 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.616563 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.618023 (+ 1460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.618519 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.618680 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.619245 (+ 565us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:52.620095 (+ 850us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:52.620249 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.620412 (+ 163us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.620512 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260811 22:09:52.628948 5762 raft_consensus.cc:1261] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Refusing update from remote peer d1adb8f1e4ae465ba20fe766bdafdd1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:52.628948 5930 raft_consensus.cc:1261] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Refusing update from remote peer d1adb8f1e4ae465ba20fe766bdafdd1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:52.629877 6021 consensus_queue.cc:1035] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:52.630095 6028 consensus_queue.cc:1035] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:52.639683 6031 mvcc.cc:205] Tried to move back new op lower bound from 7317447444999434240 to 7317447444631031808. Current Snapshot: MvccSnapshot[applied={T|T < 7317447444999434240}]
I20260811 22:09:52.653287 5384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:52.653626 5384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:52.657491 5384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9: Sending DeleteTablet for 3 replicas of tablet 03ff866918004986b5afee64b56739c7
I20260811 22:09:52.658893 5739 tablet_service.cc:1514] Processing DeleteTablet for tablet 03ff866918004986b5afee64b56739c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:52 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:38544
I20260811 22:09:52.659030 5565 tablet_service.cc:1514] Processing DeleteTablet for tablet 03ff866918004986b5afee64b56739c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:52 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:37128
I20260811 22:09:52.659621 5906 tablet_service.cc:1514] Processing DeleteTablet for tablet 03ff866918004986b5afee64b56739c7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:52 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:36180
I20260811 22:09:52.660369 6046 tablet_replica.cc:330] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f: stopping tablet replica
I20260811 22:09:52.660310 5384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58310:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:09:52.660813 5384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:52.661109 6045 tablet_replica.cc:330] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a: stopping tablet replica
I20260811 22:09:52.661724 6046 raft_consensus.cc:2229] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:52.662352 6045 raft_consensus.cc:2229] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:09:52.662675 6043 tablet_replica.cc:330] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b: stopping tablet replica
I20260811 22:09:52.662997 6043 raft_consensus.cc:2229] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:52.663367 6043 raft_consensus.cc:2258] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:52.664402 6045 raft_consensus.cc:2258] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:52.665061 6046 raft_consensus.cc:2258] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:52.666368 6043 ts_tablet_manager.cc:1905] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:52.668015 5906 tablet_service.cc:1467] Processing CreateTablet for tablet 0b495c529a06400bbd9c46c532817dd0 (DEFAULT_TABLE table=test-table [id=d30ed39ba907478cb96a6faae231231b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:52.668403 5906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b495c529a06400bbd9c46c532817dd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:52.670024 5739 tablet_service.cc:1467] Processing CreateTablet for tablet 0b495c529a06400bbd9c46c532817dd0 (DEFAULT_TABLE table=test-table [id=d30ed39ba907478cb96a6faae231231b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:52.671059 5739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b495c529a06400bbd9c46c532817dd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:52.672701 6017 tablet_bootstrap.cc:492] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f: Bootstrap starting.
I20260811 22:09:52.667613 5565 tablet_service.cc:1467] Processing CreateTablet for tablet 0b495c529a06400bbd9c46c532817dd0 (DEFAULT_TABLE table=test-table [id=d30ed39ba907478cb96a6faae231231b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:52.673470 5565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0b495c529a06400bbd9c46c532817dd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:52.673635 6016 tablet_bootstrap.cc:492] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b: Bootstrap starting.
I20260811 22:09:52.674036 6017 tablet_bootstrap.cc:654] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:52.674606 6016 tablet_bootstrap.cc:654] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:52.675711 6043 ts_tablet_manager.cc:1918] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:52.675832 6043 log.cc:1196] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/03ff866918004986b5afee64b56739c7
I20260811 22:09:52.675892 6046 ts_tablet_manager.cc:1905] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:52.676484 6015 tablet_bootstrap.cc:492] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a: Bootstrap starting.
I20260811 22:09:52.676687 6043 ts_tablet_manager.cc:1939] T 03ff866918004986b5afee64b56739c7 P 09707546d2e64cddacb610a7fcd85e6b: Deleting consensus metadata
I20260811 22:09:52.677673 6015 tablet_bootstrap.cc:654] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:52.678334 5369 catalog_manager.cc:4841] TS 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063): tablet 03ff866918004986b5afee64b56739c7 (table test-table [id=d92a9f9ee0a64849b0c252cd72a34a03]) successfully deleted
I20260811 22:09:52.679050 6015 tablet_bootstrap.cc:492] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a: No bootstrap required, opened a new log
I20260811 22:09:52.679185 6015 ts_tablet_manager.cc:1397] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:09:52.679798 6015 raft_consensus.cc:348] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.679961 6015 raft_consensus.cc:374] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:52.680007 6015 raft_consensus.cc:729] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1adb8f1e4ae465ba20fe766bdafdd1a, State: Initialized, Role: FOLLOWER
I20260811 22:09:52.680095 6045 ts_tablet_manager.cc:1905] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:52.680176 6015 consensus_queue.cc:260] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.680524 6015 ts_tablet_manager.cc:1428] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:09:52.682490 6016 tablet_bootstrap.cc:492] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b: No bootstrap required, opened a new log
I20260811 22:09:52.682601 6016 ts_tablet_manager.cc:1397] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b: Time spent bootstrapping tablet: real 0.009s user 0.002s sys 0.000s
I20260811 22:09:52.683139 6016 raft_consensus.cc:348] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.683279 6016 raft_consensus.cc:374] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:52.683324 6016 raft_consensus.cc:729] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09707546d2e64cddacb610a7fcd85e6b, State: Initialized, Role: FOLLOWER
I20260811 22:09:52.683485 6046 ts_tablet_manager.cc:1918] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:52.683473 6016 consensus_queue.cc:260] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.683594 6046 log.cc:1196] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/03ff866918004986b5afee64b56739c7
I20260811 22:09:52.683804 6045 ts_tablet_manager.cc:1918] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:52.683838 6016 ts_tablet_manager.cc:1428] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:09:52.683914 6045 log.cc:1196] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/wal/wals/03ff866918004986b5afee64b56739c7
I20260811 22:09:52.683967 6046 ts_tablet_manager.cc:1939] T 03ff866918004986b5afee64b56739c7 P fb28c8c5cf384328b2b6ced6b59ac57f: Deleting consensus metadata
I20260811 22:09:52.684325 6045 ts_tablet_manager.cc:1939] T 03ff866918004986b5afee64b56739c7 P d1adb8f1e4ae465ba20fe766bdafdd1a: Deleting consensus metadata
I20260811 22:09:52.685633 5367 catalog_manager.cc:4841] TS fb28c8c5cf384328b2b6ced6b59ac57f (127.28.47.131:38313): tablet 03ff866918004986b5afee64b56739c7 (table test-table [id=d92a9f9ee0a64849b0c252cd72a34a03]) successfully deleted
I20260811 22:09:52.685847 5367 catalog_manager.cc:4841] TS d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129:37385): tablet 03ff866918004986b5afee64b56739c7 (table test-table [id=d92a9f9ee0a64849b0c252cd72a34a03]) successfully deleted
I20260811 22:09:52.688140 6017 tablet_bootstrap.cc:492] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f: No bootstrap required, opened a new log
I20260811 22:09:52.688271 6017 ts_tablet_manager.cc:1397] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f: Time spent bootstrapping tablet: real 0.016s user 0.002s sys 0.000s
I20260811 22:09:52.689589 6017 raft_consensus.cc:348] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.689857 6017 raft_consensus.cc:374] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:52.689942 6017 raft_consensus.cc:729] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb28c8c5cf384328b2b6ced6b59ac57f, State: Initialized, Role: FOLLOWER
I20260811 22:09:52.690222 6017 consensus_queue.cc:260] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.690595 6017 ts_tablet_manager.cc:1428] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:09:52.700754 6028 raft_consensus.cc:482] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:52.700920 6028 raft_consensus.cc:504] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.701421 6028 leader_election.cc:288] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063), fb28c8c5cf384328b2b6ced6b59ac57f (127.28.47.131:38313)
I20260811 22:09:52.701961 5930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b495c529a06400bbd9c46c532817dd0" candidate_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" is_pre_election: true
I20260811 22:09:52.702039 5762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b495c529a06400bbd9c46c532817dd0" candidate_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09707546d2e64cddacb610a7fcd85e6b" is_pre_election: true
I20260811 22:09:52.702137 5930 raft_consensus.cc:2454] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1adb8f1e4ae465ba20fe766bdafdd1a in term 0.
I20260811 22:09:52.702241 5762 raft_consensus.cc:2454] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d1adb8f1e4ae465ba20fe766bdafdd1a in term 0.
I20260811 22:09:52.702541 5499 leader_election.cc:302] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1adb8f1e4ae465ba20fe766bdafdd1a, fb28c8c5cf384328b2b6ced6b59ac57f; no voters:
I20260811 22:09:52.702841 6028 raft_consensus.cc:2781] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:52.702903 6028 raft_consensus.cc:482] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:52.702945 6028 raft_consensus.cc:3037] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:52.704305 6028 raft_consensus.cc:504] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.704718 6028 leader_election.cc:288] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [CANDIDATE]: Term 1 election: Requested vote from peers 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063), fb28c8c5cf384328b2b6ced6b59ac57f (127.28.47.131:38313)
I20260811 22:09:52.705420 5930 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b495c529a06400bbd9c46c532817dd0" candidate_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f"
I20260811 22:09:52.705565 5930 raft_consensus.cc:3037] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:52.706532 5762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0b495c529a06400bbd9c46c532817dd0" candidate_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09707546d2e64cddacb610a7fcd85e6b"
I20260811 22:09:52.706717 5762 raft_consensus.cc:3037] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:52.706875 5930 raft_consensus.cc:2454] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1adb8f1e4ae465ba20fe766bdafdd1a in term 1.
I20260811 22:09:52.707242 5499 leader_election.cc:302] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1adb8f1e4ae465ba20fe766bdafdd1a, fb28c8c5cf384328b2b6ced6b59ac57f; no voters:
I20260811 22:09:52.707453 6028 raft_consensus.cc:2781] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:52.707537 6028 raft_consensus.cc:686] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 LEADER]: Becoming Leader. State: Replica: d1adb8f1e4ae465ba20fe766bdafdd1a, State: Running, Role: LEADER
I20260811 22:09:52.707685 6028 consensus_queue.cc:237] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.708343 5762 raft_consensus.cc:2454] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d1adb8f1e4ae465ba20fe766bdafdd1a in term 1.
I20260811 22:09:52.710402 5384 catalog_manager.cc:5495] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a reported cstate change: term changed from 0 to 1, leader changed from <none> to d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129). New cstate: current_term: 1 leader_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } health_report { overall_health: UNKNOWN } } }
Aug 11 22:09:52 dist-test-slave-vxj2 krb5kdc[5331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486191, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.190@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:52.764340 6060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.751792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58326 (local address 127.28.47.190:38251)
0811 22:09:52.752174 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.752187 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.752292 (+ 105us) server_negotiation.cc:407] Connection header received
0811 22:09:52.752443 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.752448 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.752518 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.752645 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:52.753620 (+ 975us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.754640 (+ 1020us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.755879 (+ 1239us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.756387 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.756539 (+ 152us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.759986 (+ 3447us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:52.759990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:52.759993 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:52.760021 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:52.761918 (+ 1897us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:52.762625 (+ 707us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:52.762629 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:52.762631 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:52.762723 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:52.763060 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:52.763064 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:52.763065 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:52.763303 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:52.763399 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.763861 (+ 462us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.764117 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":95,"server-negotiator.queue_time_us":273,"thread_start_us":137,"threads_started":1}
I20260811 22:09:52.764353 6059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.751678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38251 (local address 127.0.0.1:58326)
0811 22:09:52.752184 (+ 506us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:52.752201 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:52.752329 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:52.752692 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:52.752696 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:52.753295 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:52.753443 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.753452 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.754804 (+ 1352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.754808 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.755710 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.755722 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.756538 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.756542 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.756659 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.757551 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:52.757567 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:52.759786 (+ 2219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:52.762069 (+ 2283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:52.762084 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:52.762094 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:52.762497 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:52.762832 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:52.762838 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:52.762843 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:52.762959 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:52.763419 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:52.763430 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:52.763615 (+ 185us) client_negotiation.cc:769] Sending connection context
0811 22:09:52.763777 (+ 162us) client_negotiation.cc:240] Negotiation successful
0811 22:09:52.764007 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":5,"thread_start_us":153,"threads_started":1}
I20260811 22:09:52.770386 5384 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:58326:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260811 22:09:52.771310 5384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:52.776822 5565 tablet_service.cc:1467] Processing CreateTablet for tablet 8e101e8bdb9c43a5ba2c91cdbc5688fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ac5d015f982940f2bbbf80d93a6b2131]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:09:52.777128 5906 tablet_service.cc:1467] Processing CreateTablet for tablet 8e101e8bdb9c43a5ba2c91cdbc5688fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ac5d015f982940f2bbbf80d93a6b2131]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:09:52.777128 5739 tablet_service.cc:1467] Processing CreateTablet for tablet 8e101e8bdb9c43a5ba2c91cdbc5688fb (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=ac5d015f982940f2bbbf80d93a6b2131]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260811 22:09:52.777324 5565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e101e8bdb9c43a5ba2c91cdbc5688fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:52.777529 5739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e101e8bdb9c43a5ba2c91cdbc5688fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:52.777529 5906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e101e8bdb9c43a5ba2c91cdbc5688fb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:52.780473 6015 tablet_bootstrap.cc:492] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a: Bootstrap starting.
I20260811 22:09:52.780530 6016 tablet_bootstrap.cc:492] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b: Bootstrap starting.
I20260811 22:09:52.781612 6016 tablet_bootstrap.cc:654] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:52.781921 6015 tablet_bootstrap.cc:654] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:52.783274 6016 tablet_bootstrap.cc:492] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b: No bootstrap required, opened a new log
I20260811 22:09:52.783384 6016 ts_tablet_manager.cc:1397] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260811 22:09:52.783806 6016 raft_consensus.cc:348] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.783938 6016 raft_consensus.cc:374] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:52.783986 6016 raft_consensus.cc:729] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09707546d2e64cddacb610a7fcd85e6b, State: Initialized, Role: FOLLOWER
I20260811 22:09:52.784130 6016 consensus_queue.cc:260] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.784433 6016 ts_tablet_manager.cc:1428] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:09:52.784390 6023 tablet_replica.cc:439] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } }
I20260811 22:09:52.784536 6023 tablet_replica.cc:442] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:52.784734 6017 tablet_bootstrap.cc:492] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f: Bootstrap starting.
I20260811 22:09:52.786242 6017 tablet_bootstrap.cc:654] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:52.787683 6017 tablet_bootstrap.cc:492] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f: No bootstrap required, opened a new log
I20260811 22:09:52.787719 6015 tablet_bootstrap.cc:492] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a: No bootstrap required, opened a new log
I20260811 22:09:52.787819 6015 ts_tablet_manager.cc:1397] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.000s
I20260811 22:09:52.787817 6017 ts_tablet_manager.cc:1397] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:52.788271 6017 raft_consensus.cc:348] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.788304 6015 raft_consensus.cc:348] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.788460 6015 raft_consensus.cc:374] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:52.788476 6017 raft_consensus.cc:374] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:52.788492 6015 raft_consensus.cc:729] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1adb8f1e4ae465ba20fe766bdafdd1a, State: Initialized, Role: FOLLOWER
I20260811 22:09:52.788525 6017 raft_consensus.cc:729] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb28c8c5cf384328b2b6ced6b59ac57f, State: Initialized, Role: FOLLOWER
I20260811 22:09:52.788710 6017 consensus_queue.cc:260] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.788731 6015 consensus_queue.cc:260] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.789008 6021 tablet_replica.cc:439] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } }
I20260811 22:09:52.789012 6022 tablet_replica.cc:439] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } }
I20260811 22:09:52.789172 6021 tablet_replica.cc:442] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:52.789182 6022 tablet_replica.cc:442] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:52.789623 6015 ts_tablet_manager.cc:1428] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:09:52.790179 6017 ts_tablet_manager.cc:1428] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260811 22:09:52.805133 6022 raft_consensus.cc:482] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:52.805322 6022 raft_consensus.cc:504] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.806427 6022 leader_election.cc:288] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129:37385), 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063)
I20260811 22:09:52.811374 6010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.806841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:52747 (local address 127.28.47.129:37385)
0811 22:09:52.806975 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.806980 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.807433 (+ 453us) server_negotiation.cc:407] Connection header received
0811 22:09:52.807552 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.807556 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.807609 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.807682 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:52.808024 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.808743 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.810235 (+ 1492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.810745 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.810823 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.810873 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.811171 (+ 298us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.811250 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:09:52.811410 6066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.806659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:37385 (local address 127.28.47.131:52747)
0811 22:09:52.807331 (+ 672us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:52.807343 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:52.807467 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:52.807724 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:52.807728 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:52.807750 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:52.807892 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.807899 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.808851 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.808854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.810088 (+ 1234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.810099 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.810958 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.810960 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.811042 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.811059 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 22:09:52.811149 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:09:52.811227 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":608,"mutex_wait_us":372,"thread_start_us":110,"threads_started":1}
I20260811 22:09:52.812116 6013 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.806816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:56111 (local address 127.28.47.130:38063)
0811 22:09:52.807002 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.807007 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.807137 (+ 130us) server_negotiation.cc:407] Connection header received
0811 22:09:52.807317 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.807320 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.807389 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.807473 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:09:52.807838 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.808618 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.810304 (+ 1686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.810910 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.811010 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.811073 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.811903 (+ 830us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.811995 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260811 22:09:52.812062 5584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e101e8bdb9c43a5ba2c91cdbc5688fb" candidate_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" is_pre_election: true
I20260811 22:09:52.812348 5584 raft_consensus.cc:2454] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb28c8c5cf384328b2b6ced6b59ac57f in term 0.
I20260811 22:09:52.812590 6065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.806658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:38063 (local address 127.28.47.131:56111)
0811 22:09:52.807034 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:52.807057 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:52.807191 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:52.807574 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:52.807577 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:52.807589 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:09:52.807737 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.807746 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.808757 (+ 1011us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.808761 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.810143 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.810155 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.811218 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.811223 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.811352 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.811385 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:09:52.812340 (+ 955us) client_negotiation.cc:240] Negotiation successful
0811 22:09:52.812421 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":29,"thread_start_us":125,"threads_started":1}
I20260811 22:09:52.812824 5847 leader_election.cc:302] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1adb8f1e4ae465ba20fe766bdafdd1a, fb28c8c5cf384328b2b6ced6b59ac57f; no voters:
I20260811 22:09:52.813081 5762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e101e8bdb9c43a5ba2c91cdbc5688fb" candidate_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09707546d2e64cddacb610a7fcd85e6b" is_pre_election: true
I20260811 22:09:52.813199 6022 raft_consensus.cc:2781] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:52.813252 5762 raft_consensus.cc:2454] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb28c8c5cf384328b2b6ced6b59ac57f in term 0.
I20260811 22:09:52.813264 6022 raft_consensus.cc:482] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:52.813306 6022 raft_consensus.cc:3037] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:52.814642 6022 raft_consensus.cc:504] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.815080 6022 leader_election.cc:288] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [CANDIDATE]: Term 1 election: Requested vote from peers d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129:37385), 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063)
I20260811 22:09:52.815507 5584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e101e8bdb9c43a5ba2c91cdbc5688fb" candidate_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a"
I20260811 22:09:52.815653 5584 raft_consensus.cc:3037] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:52.815631 5762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e101e8bdb9c43a5ba2c91cdbc5688fb" candidate_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09707546d2e64cddacb610a7fcd85e6b"
I20260811 22:09:52.815785 5762 raft_consensus.cc:3037] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:52.816992 5584 raft_consensus.cc:2454] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb28c8c5cf384328b2b6ced6b59ac57f in term 1.
I20260811 22:09:52.817075 5762 raft_consensus.cc:2454] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb28c8c5cf384328b2b6ced6b59ac57f in term 1.
I20260811 22:09:52.817394 5847 leader_election.cc:302] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1adb8f1e4ae465ba20fe766bdafdd1a, fb28c8c5cf384328b2b6ced6b59ac57f; no voters:
I20260811 22:09:52.817648 6022 raft_consensus.cc:2781] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:52.817914 6022 raft_consensus.cc:686] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 LEADER]: Becoming Leader. State: Replica: fb28c8c5cf384328b2b6ced6b59ac57f, State: Running, Role: LEADER
I20260811 22:09:52.818218 6022 consensus_queue.cc:237] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.819694 6067 tablet_replica.cc:439] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f: TxnStatusTablet state changed. Reason: New leader fb28c8c5cf384328b2b6ced6b59ac57f. Latest consensus state: current_term: 1 leader_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } }
I20260811 22:09:52.819877 6067 tablet_replica.cc:442] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:09:52.820969 6069 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260811 22:09:52.821182 6069 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260811 22:09:52.821041 5382 catalog_manager.cc:5495] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f reported cstate change: term changed from 0 to 1, leader changed from <none> to fb28c8c5cf384328b2b6ced6b59ac57f (127.28.47.131). New cstate: current_term: 1 leader_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } health_report { overall_health: HEALTHY } } }
I20260811 22:09:52.842213 6059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.835844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:38313 (local address 127.0.0.1:36196)
0811 22:09:52.836224 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:52.836241 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:52.836367 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:52.837337 (+ 970us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:52.837340 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:52.837350 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:09:52.837480 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.837487 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.838669 (+ 1182us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.838673 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.839690 (+ 1017us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.839710 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.840428 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.840433 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.840556 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.840629 (+ 73us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:09:52.841802 (+ 1173us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:09:52.841823 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:09:52.841906 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:09:52.842026 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":241}
I20260811 22:09:52.843081 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.836649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36196 (local address 127.28.47.131:38313)
0811 22:09:52.836805 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.836813 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.836833 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:09:52.836941 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.836945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.837020 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.837123 (+ 103us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:52.837649 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.838487 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.839855 (+ 1368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.840230 (+ 375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.840329 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.840856 (+ 527us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:52.841611 (+ 755us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:52.841739 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.842867 (+ 1128us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.842943 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:09:52.846733 5762 raft_consensus.cc:1261] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Refusing update from remote peer fb28c8c5cf384328b2b6ced6b59ac57f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:52.846926 5584 raft_consensus.cc:1261] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Refusing update from remote peer fb28c8c5cf384328b2b6ced6b59ac57f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:52.847371 6067 consensus_queue.cc:1035] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [LEADER]: Connected to new peer: Peer: permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:52.847759 6022 consensus_queue.cc:1035] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:52.849220 6023 tablet_replica.cc:439] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b: TxnStatusTablet state changed. Reason: New leader fb28c8c5cf384328b2b6ced6b59ac57f. Latest consensus state: current_term: 1 leader_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } }
I20260811 22:09:52.849350 6023 tablet_replica.cc:442] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:52.850849 6021 tablet_replica.cc:439] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a: TxnStatusTablet state changed. Reason: New leader fb28c8c5cf384328b2b6ced6b59ac57f. Latest consensus state: current_term: 1 leader_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } }
I20260811 22:09:52.850996 6021 tablet_replica.cc:442] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:52.856032 6032 mvcc.cc:205] Tried to move back new op lower bound from 7317447445894197248 to 7317447445786730496. Current Snapshot: MvccSnapshot[applied={T|T < 7317447445894197248}]
I20260811 22:09:52.858937 6023 tablet_replica.cc:439] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } }
I20260811 22:09:52.859099 6023 tablet_replica.cc:442] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P 09707546d2e64cddacb610a7fcd85e6b: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:52.861294 6067 tablet_replica.cc:439] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } }
I20260811 22:09:52.861544 6067 tablet_replica.cc:442] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:09:52.862914 6067 tablet_replica.cc:439] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } }
I20260811 22:09:52.863126 6067 tablet_replica.cc:442] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P fb28c8c5cf384328b2b6ced6b59ac57f: This TxnStatusTablet replica's current role is: LEADER
I20260811 22:09:52.863684 6021 tablet_replica.cc:439] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } } }
I20260811 22:09:52.863813 6021 tablet_replica.cc:442] T 8e101e8bdb9c43a5ba2c91cdbc5688fb P d1adb8f1e4ae465ba20fe766bdafdd1a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260811 22:09:52.876183 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.872221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:48035 (local address 127.28.47.131:38313)
0811 22:09:52.872397 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.872403 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.872633 (+ 230us) server_negotiation.cc:407] Connection header received
0811 22:09:52.872746 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.872749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.872807 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.872885 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:52.873222 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.873749 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.875035 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.875262 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.875350 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.875728 (+ 378us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:52.875826 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:52.875918 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.876007 (+ 89us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.876059 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:09:52.876281 6078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.872100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:38313 (local address 127.28.47.129:48035)
0811 22:09:52.872504 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:52.872520 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:52.872652 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:52.872928 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:52.872931 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:52.872940 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:09:52.873064 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.873071 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.873887 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.873891 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.874896 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.874911 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.875387 (+ 476us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.875391 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.875485 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.875552 (+ 67us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:09:52.875926 (+ 374us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:09:52.875933 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:09:52.876000 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:09:52.876111 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":168,"threads_started":1}
I20260811 22:09:52.884789 5930 raft_consensus.cc:1261] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Refusing update from remote peer d1adb8f1e4ae465ba20fe766bdafdd1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:52.884785 5762 raft_consensus.cc:1261] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Refusing update from remote peer d1adb8f1e4ae465ba20fe766bdafdd1a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:52.885272 6028 consensus_queue.cc:1035] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:52.885440 6021 consensus_queue.cc:1035] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [LEADER]: Connected to new peer: Peer: permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:52.908934 6010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.904902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:59699 (local address 127.28.47.129:37385)
0811 22:09:52.905038 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.905047 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.905389 (+ 342us) server_negotiation.cc:407] Connection header received
0811 22:09:52.905556 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.905559 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.905608 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.905686 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:52.906010 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.906655 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.907713 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.907965 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.908079 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.908408 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:52.908495 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:52.908574 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.908682 (+ 108us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.908745 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:09:52.908943 6087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.904814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:37385 (local address 127.28.47.131:59699)
0811 22:09:52.905284 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:52.905298 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:52.905442 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:52.905706 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:52.905710 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:52.905719 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:09:52.905860 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.905868 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.906772 (+ 904us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.906777 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.907583 (+ 806us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:52.907594 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.908092 (+ 498us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.908095 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:52.908194 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.908244 (+ 50us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:09:52.908589 (+ 345us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:09:52.908596 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:09:52.908666 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 22:09:52.908757 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":114,"threads_started":1}
I20260811 22:09:52.935950 5384 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:52.936113 5384 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:58310:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:52.938598 5384 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9: Sending DeleteTablet for 3 replicas of tablet 0b495c529a06400bbd9c46c532817dd0
I20260811 22:09:52.939284 5565 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b495c529a06400bbd9c46c532817dd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:52 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:37128
I20260811 22:09:52.939474 6045 tablet_replica.cc:330] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a: stopping tablet replica
I20260811 22:09:52.939450 5739 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b495c529a06400bbd9c46c532817dd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:52 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:38544
I20260811 22:09:52.939590 6045 raft_consensus.cc:2229] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:09:52.939780 6045 raft_consensus.cc:2258] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:52.939961 6043 tablet_replica.cc:330] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b: stopping tablet replica
I20260811 22:09:52.940068 6043 raft_consensus.cc:2229] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:52.940172 6043 raft_consensus.cc:2258] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:52.940600 6045 ts_tablet_manager.cc:1905] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:52.940642 5906 tablet_service.cc:1514] Processing DeleteTablet for tablet 0b495c529a06400bbd9c46c532817dd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:52 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:36180
I20260811 22:09:52.940819 6046 tablet_replica.cc:330] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f: stopping tablet replica
I20260811 22:09:52.940918 6046 raft_consensus.cc:2229] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:52.941025 6046 raft_consensus.cc:2258] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:52.941949 6046 ts_tablet_manager.cc:1905] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:52.944588 6045 ts_tablet_manager.cc:1918] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:52.944687 6045 log.cc:1196] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/wal/wals/0b495c529a06400bbd9c46c532817dd0
I20260811 22:09:52.944696 6046 ts_tablet_manager.cc:1918] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:52.944785 6046 log.cc:1196] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/0b495c529a06400bbd9c46c532817dd0
I20260811 22:09:52.944972 6045 ts_tablet_manager.cc:1939] T 0b495c529a06400bbd9c46c532817dd0 P d1adb8f1e4ae465ba20fe766bdafdd1a: Deleting consensus metadata
I20260811 22:09:52.945041 6046 ts_tablet_manager.cc:1939] T 0b495c529a06400bbd9c46c532817dd0 P fb28c8c5cf384328b2b6ced6b59ac57f: Deleting consensus metadata
I20260811 22:09:52.946027 5367 catalog_manager.cc:4841] TS d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129:37385): tablet 0b495c529a06400bbd9c46c532817dd0 (table test-table [id=d30ed39ba907478cb96a6faae231231b]) successfully deleted
I20260811 22:09:52.946297 5367 catalog_manager.cc:4841] TS fb28c8c5cf384328b2b6ced6b59ac57f (127.28.47.131:38313): tablet 0b495c529a06400bbd9c46c532817dd0 (table test-table [id=d30ed39ba907478cb96a6faae231231b]) successfully deleted
I20260811 22:09:52.946826 6043 ts_tablet_manager.cc:1905] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:52.949971 6043 ts_tablet_manager.cc:1918] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:52.950060 6043 log.cc:1196] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/0b495c529a06400bbd9c46c532817dd0
I20260811 22:09:52.950330 6043 ts_tablet_manager.cc:1939] T 0b495c529a06400bbd9c46c532817dd0 P 09707546d2e64cddacb610a7fcd85e6b: Deleting consensus metadata
I20260811 22:09:52.951655 5369 catalog_manager.cc:4841] TS 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063): tablet 0b495c529a06400bbd9c46c532817dd0 (table test-table [id=d30ed39ba907478cb96a6faae231231b]) successfully deleted
Aug 11 22:09:52 dist-test-slave-vxj2 krb5kdc[5331](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 11 22:09:52 dist-test-slave-vxj2 krb5kdc[5331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:52 dist-test-slave-vxj2 krb5kdc[5331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:09:52.971483 6060 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:52.964252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58332 (local address 127.28.47.190:38251)
0811 22:09:52.964411 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.964415 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.964563 (+ 148us) server_negotiation.cc:407] Connection header received
0811 22:09:52.964662 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.964665 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.964721 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.964830 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:52.965853 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.966466 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.967384 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.967666 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.967785 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.971327 (+ 3542us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:58332: BlockingRecv error: recv got EOF from 127.0.0.1:58332 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 22:09:52 dist-test-slave-vxj2 krb5kdc[5331](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM
I20260811 22:09:52.984252 6060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:52.973768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58344 (local address 127.28.47.190:38251)
0811 22:09:52.973943 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:52.973947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:52.974132 (+ 185us) server_negotiation.cc:407] Connection header received
0811 22:09:52.974287 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:52.974289 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:52.974330 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:52.974404 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:52.975246 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.975810 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.977076 (+ 1266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:52.977391 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:52.977481 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:52.980389 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:52.980392 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:52.980395 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:52.980425 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:52.982169 (+ 1744us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:52.982734 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:52.982737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:52.982738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:52.982837 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:52.983259 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:52.983262 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:52.983263 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:52.983557 (+ 294us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:52.983678 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:52.983972 (+ 294us) server_negotiation.cc:299] Negotiation successful
0811 22:09:52.984086 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:09:52.987387 5384 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:09:52.987915 5384 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:52.993846 5906 tablet_service.cc:1467] Processing CreateTablet for tablet a18ee8468ea54046851853c3f24bfe0f (DEFAULT_TABLE table=test-table [id=34aae4873cb04ae7bbd488832d7b6594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:52.993743 5565 tablet_service.cc:1467] Processing CreateTablet for tablet a18ee8468ea54046851853c3f24bfe0f (DEFAULT_TABLE table=test-table [id=34aae4873cb04ae7bbd488832d7b6594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:52.994221 5565 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a18ee8468ea54046851853c3f24bfe0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:52.994246 5906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a18ee8468ea54046851853c3f24bfe0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:52.996568 6017 tablet_bootstrap.cc:492] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f: Bootstrap starting.
I20260811 22:09:52.996827 5739 tablet_service.cc:1467] Processing CreateTablet for tablet a18ee8468ea54046851853c3f24bfe0f (DEFAULT_TABLE table=test-table [id=34aae4873cb04ae7bbd488832d7b6594]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:52.996963 6015 tablet_bootstrap.cc:492] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a: Bootstrap starting.
I20260811 22:09:52.997133 5739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a18ee8468ea54046851853c3f24bfe0f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:52.997632 6017 tablet_bootstrap.cc:654] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:52.997920 6015 tablet_bootstrap.cc:654] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:52.999248 6015 tablet_bootstrap.cc:492] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a: No bootstrap required, opened a new log
I20260811 22:09:52.999347 6015 ts_tablet_manager.cc:1397] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:09:52.999709 6016 tablet_bootstrap.cc:492] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b: Bootstrap starting.
I20260811 22:09:52.999758 6015 raft_consensus.cc:348] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:52.999881 6015 raft_consensus.cc:374] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:52.999917 6015 raft_consensus.cc:729] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1adb8f1e4ae465ba20fe766bdafdd1a, State: Initialized, Role: FOLLOWER
I20260811 22:09:53.000049 6015 consensus_queue.cc:260] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.000326 6015 ts_tablet_manager.cc:1428] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:09:53.000705 6016 tablet_bootstrap.cc:654] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:53.001359 6017 tablet_bootstrap.cc:492] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f: No bootstrap required, opened a new log
I20260811 22:09:53.001452 6017 ts_tablet_manager.cc:1397] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260811 22:09:53.001962 6017 raft_consensus.cc:348] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.002084 6017 raft_consensus.cc:374] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:53.002120 6017 raft_consensus.cc:729] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb28c8c5cf384328b2b6ced6b59ac57f, State: Initialized, Role: FOLLOWER
I20260811 22:09:53.002291 6017 consensus_queue.cc:260] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.003304 6016 tablet_bootstrap.cc:492] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b: No bootstrap required, opened a new log
I20260811 22:09:53.003423 6016 ts_tablet_manager.cc:1397] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260811 22:09:53.003969 6017 ts_tablet_manager.cc:1428] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f: Time spent starting tablet: real 0.002s user 0.001s sys 0.001s
I20260811 22:09:53.003921 6016 raft_consensus.cc:348] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.004043 6016 raft_consensus.cc:374] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:53.004077 6016 raft_consensus.cc:729] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09707546d2e64cddacb610a7fcd85e6b, State: Initialized, Role: FOLLOWER
I20260811 22:09:53.004210 6016 consensus_queue.cc:260] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.004521 6016 ts_tablet_manager.cc:1428] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:09:53.307787 6084 raft_consensus.cc:482] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:53.308002 6084 raft_consensus.cc:504] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.308537 6084 leader_election.cc:288] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129:37385), 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063)
I20260811 22:09:53.309082 5584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a18ee8468ea54046851853c3f24bfe0f" candidate_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" is_pre_election: true
I20260811 22:09:53.309091 5762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a18ee8468ea54046851853c3f24bfe0f" candidate_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09707546d2e64cddacb610a7fcd85e6b" is_pre_election: true
I20260811 22:09:53.309294 5584 raft_consensus.cc:2454] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb28c8c5cf384328b2b6ced6b59ac57f in term 0.
I20260811 22:09:53.309311 5762 raft_consensus.cc:2454] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb28c8c5cf384328b2b6ced6b59ac57f in term 0.
I20260811 22:09:53.309676 5850 leader_election.cc:302] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09707546d2e64cddacb610a7fcd85e6b, fb28c8c5cf384328b2b6ced6b59ac57f; no voters:
I20260811 22:09:53.309903 6084 raft_consensus.cc:2781] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:53.309971 6084 raft_consensus.cc:482] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:53.310009 6084 raft_consensus.cc:3037] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:53.311059 6084 raft_consensus.cc:504] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.311474 6084 leader_election.cc:288] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [CANDIDATE]: Term 1 election: Requested vote from peers d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129:37385), 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063)
I20260811 22:09:53.311973 5584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a18ee8468ea54046851853c3f24bfe0f" candidate_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a"
I20260811 22:09:53.311966 5762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a18ee8468ea54046851853c3f24bfe0f" candidate_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09707546d2e64cddacb610a7fcd85e6b"
I20260811 22:09:53.312136 5584 raft_consensus.cc:3037] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:53.312151 5762 raft_consensus.cc:3037] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:53.313143 5762 raft_consensus.cc:2454] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb28c8c5cf384328b2b6ced6b59ac57f in term 1.
I20260811 22:09:53.313383 5584 raft_consensus.cc:2454] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb28c8c5cf384328b2b6ced6b59ac57f in term 1.
I20260811 22:09:53.313720 5850 leader_election.cc:302] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09707546d2e64cddacb610a7fcd85e6b, fb28c8c5cf384328b2b6ced6b59ac57f; no voters:
I20260811 22:09:53.313949 6084 raft_consensus.cc:2781] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:53.314077 6084 raft_consensus.cc:686] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 LEADER]: Becoming Leader. State: Replica: fb28c8c5cf384328b2b6ced6b59ac57f, State: Running, Role: LEADER
I20260811 22:09:53.314276 6084 consensus_queue.cc:237] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.316202 5384 catalog_manager.cc:5495] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f reported cstate change: term changed from 0 to 1, leader changed from <none> to fb28c8c5cf384328b2b6ced6b59ac57f (127.28.47.131). New cstate: current_term: 1 leader_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } health_report { overall_health: HEALTHY } } }
I20260811 22:09:53.373996 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:53.370228 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36212 (local address 127.28.47.131:38313)
0811 22:09:53.370389 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:53.370396 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:53.370413 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:09:53.370627 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:53.370630 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:53.370696 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:53.370789 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:53.371090 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:53.371599 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:53.372638 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:53.372877 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:53.373022 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:53.373473 (+ 451us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:53.373560 (+ 87us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:53.373629 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:53.373750 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 22:09:53.373805 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260811 22:09:53.376713 5762 raft_consensus.cc:1261] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Refusing update from remote peer fb28c8c5cf384328b2b6ced6b59ac57f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:53.376791 5584 raft_consensus.cc:1261] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Refusing update from remote peer fb28c8c5cf384328b2b6ced6b59ac57f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:53.377317 6067 consensus_queue.cc:1035] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [LEADER]: Connected to new peer: Peer: permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:53.377533 6084 consensus_queue.cc:1035] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:53.390959 5383 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:53.391144 5383 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:53.393837 5383 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9: Sending DeleteTablet for 3 replicas of tablet a18ee8468ea54046851853c3f24bfe0f
I20260811 22:09:53.394492 5739 tablet_service.cc:1514] Processing DeleteTablet for tablet a18ee8468ea54046851853c3f24bfe0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:53 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:38544
I20260811 22:09:53.394640 5906 tablet_service.cc:1514] Processing DeleteTablet for tablet a18ee8468ea54046851853c3f24bfe0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:53 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:36180
I20260811 22:09:53.394675 5565 tablet_service.cc:1514] Processing DeleteTablet for tablet a18ee8468ea54046851853c3f24bfe0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:53 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:37128
I20260811 22:09:53.394855 6046 tablet_replica.cc:330] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f: stopping tablet replica
I20260811 22:09:53.394877 5564 tablet_service.cc:1514] Processing DeleteTablet for tablet a18ee8468ea54046851853c3f24bfe0f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:53 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:37128
I20260811 22:09:53.394897 6043 tablet_replica.cc:330] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b: stopping tablet replica
I20260811 22:09:53.394986 6046 raft_consensus.cc:2229] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:09:53.395028 6045 tablet_replica.cc:330] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a: stopping tablet replica
I20260811 22:09:53.395028 6043 raft_consensus.cc:2229] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:53.395131 6045 raft_consensus.cc:2229] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:53.395166 6043 raft_consensus.cc:2258] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:53.395248 6045 raft_consensus.cc:2258] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:53.395327 6046 raft_consensus.cc:2258] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:53.396000 6043 ts_tablet_manager.cc:1905] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:53.396054 6046 ts_tablet_manager.cc:1905] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:53.396659 5383 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:09:53.397110 5383 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:09:53.398689 6045 ts_tablet_manager.cc:1905] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:53.398813 6043 ts_tablet_manager.cc:1918] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:53.398819 6046 ts_tablet_manager.cc:1918] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:53.398891 6043 log.cc:1196] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/a18ee8468ea54046851853c3f24bfe0f
I20260811 22:09:53.398893 6046 log.cc:1196] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/a18ee8468ea54046851853c3f24bfe0f
I20260811 22:09:53.399170 6043 ts_tablet_manager.cc:1939] T a18ee8468ea54046851853c3f24bfe0f P 09707546d2e64cddacb610a7fcd85e6b: Deleting consensus metadata
I20260811 22:09:53.399173 6046 ts_tablet_manager.cc:1939] T a18ee8468ea54046851853c3f24bfe0f P fb28c8c5cf384328b2b6ced6b59ac57f: Deleting consensus metadata
I20260811 22:09:53.401098 5369 catalog_manager.cc:4841] TS 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063): tablet a18ee8468ea54046851853c3f24bfe0f (table test-table [id=34aae4873cb04ae7bbd488832d7b6594]) successfully deleted
I20260811 22:09:53.401804 6045 ts_tablet_manager.cc:1918] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:09:53.401912 6045 log.cc:1196] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-0/wal/wals/a18ee8468ea54046851853c3f24bfe0f
I20260811 22:09:53.401990 5367 catalog_manager.cc:4841] TS fb28c8c5cf384328b2b6ced6b59ac57f (127.28.47.131:38313): tablet a18ee8468ea54046851853c3f24bfe0f (table test-table [id=34aae4873cb04ae7bbd488832d7b6594]) successfully deleted
I20260811 22:09:53.402230 6045 ts_tablet_manager.cc:1939] T a18ee8468ea54046851853c3f24bfe0f P d1adb8f1e4ae465ba20fe766bdafdd1a: Deleting consensus metadata
I20260811 22:09:53.403281 5367 catalog_manager.cc:4841] TS d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129:37385): tablet a18ee8468ea54046851853c3f24bfe0f (table test-table [id=34aae4873cb04ae7bbd488832d7b6594]) successfully deleted
W20260811 22:09:53.403601 5367 catalog_manager.cc:4808] TS d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129:37385): delete failed for tablet a18ee8468ea54046851853c3f24bfe0f because the tablet was not found. No further retry: Not found: Tablet not found: a18ee8468ea54046851853c3f24bfe0f
I20260811 22:09:53.405385 5564 tablet_service.cc:1467] Processing CreateTablet for tablet 54d90f76f24d4836b9dee3886edf27e0 (DEFAULT_TABLE table=test-table [id=b88731c072de461980322c43354ce1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:53.405646 5906 tablet_service.cc:1467] Processing CreateTablet for tablet 54d90f76f24d4836b9dee3886edf27e0 (DEFAULT_TABLE table=test-table [id=b88731c072de461980322c43354ce1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:53.405742 5564 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54d90f76f24d4836b9dee3886edf27e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:53.405946 5906 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54d90f76f24d4836b9dee3886edf27e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:53.408113 6015 tablet_bootstrap.cc:492] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a: Bootstrap starting.
I20260811 22:09:53.408491 6017 tablet_bootstrap.cc:492] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f: Bootstrap starting.
I20260811 22:09:53.409127 6015 tablet_bootstrap.cc:654] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:53.409473 6017 tablet_bootstrap.cc:654] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:53.410292 5739 tablet_service.cc:1467] Processing CreateTablet for tablet 54d90f76f24d4836b9dee3886edf27e0 (DEFAULT_TABLE table=test-table [id=b88731c072de461980322c43354ce1c1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:09:53.410606 5739 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54d90f76f24d4836b9dee3886edf27e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:53.410658 6017 tablet_bootstrap.cc:492] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f: No bootstrap required, opened a new log
I20260811 22:09:53.410758 6017 ts_tablet_manager.cc:1397] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:09:53.411172 6017 raft_consensus.cc:348] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.411310 6017 raft_consensus.cc:374] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:53.411343 6017 raft_consensus.cc:729] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fb28c8c5cf384328b2b6ced6b59ac57f, State: Initialized, Role: FOLLOWER
I20260811 22:09:53.411471 6017 consensus_queue.cc:260] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.411732 6017 ts_tablet_manager.cc:1428] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260811 22:09:53.412951 6016 tablet_bootstrap.cc:492] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b: Bootstrap starting.
I20260811 22:09:53.414287 6016 tablet_bootstrap.cc:654] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:53.414378 6015 tablet_bootstrap.cc:492] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a: No bootstrap required, opened a new log
I20260811 22:09:53.414489 6015 ts_tablet_manager.cc:1397] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260811 22:09:53.414959 6015 raft_consensus.cc:348] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.415136 6015 raft_consensus.cc:374] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:53.415184 6015 raft_consensus.cc:729] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d1adb8f1e4ae465ba20fe766bdafdd1a, State: Initialized, Role: FOLLOWER
I20260811 22:09:53.415333 6015 consensus_queue.cc:260] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.415628 6015 ts_tablet_manager.cc:1428] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260811 22:09:53.416055 6016 tablet_bootstrap.cc:492] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b: No bootstrap required, opened a new log
I20260811 22:09:53.416148 6016 ts_tablet_manager.cc:1397] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:53.416580 6016 raft_consensus.cc:348] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.416687 6016 raft_consensus.cc:374] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:53.416718 6016 raft_consensus.cc:729] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09707546d2e64cddacb610a7fcd85e6b, State: Initialized, Role: FOLLOWER
I20260811 22:09:53.416836 6016 consensus_queue.cc:260] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.419175 6016 ts_tablet_manager.cc:1428] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260811 22:09:53.523525 6084 raft_consensus.cc:482] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:09:53.523732 6084 raft_consensus.cc:504] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.524264 6084 leader_election.cc:288] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129:37385), 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063)
I20260811 22:09:53.524772 5584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54d90f76f24d4836b9dee3886edf27e0" candidate_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" is_pre_election: true
I20260811 22:09:53.524770 5762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54d90f76f24d4836b9dee3886edf27e0" candidate_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09707546d2e64cddacb610a7fcd85e6b" is_pre_election: true
I20260811 22:09:53.524951 5762 raft_consensus.cc:2454] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb28c8c5cf384328b2b6ced6b59ac57f in term 0.
I20260811 22:09:53.524952 5584 raft_consensus.cc:2454] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fb28c8c5cf384328b2b6ced6b59ac57f in term 0.
I20260811 22:09:53.525328 5850 leader_election.cc:302] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09707546d2e64cddacb610a7fcd85e6b, fb28c8c5cf384328b2b6ced6b59ac57f; no voters:
I20260811 22:09:53.525570 6084 raft_consensus.cc:2781] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:09:53.525657 6084 raft_consensus.cc:482] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:09:53.525715 6084 raft_consensus.cc:3037] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:53.526834 6084 raft_consensus.cc:504] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.527280 6084 leader_election.cc:288] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [CANDIDATE]: Term 1 election: Requested vote from peers d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129:37385), 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063)
I20260811 22:09:53.527715 5762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54d90f76f24d4836b9dee3886edf27e0" candidate_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09707546d2e64cddacb610a7fcd85e6b"
I20260811 22:09:53.527734 5584 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54d90f76f24d4836b9dee3886edf27e0" candidate_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a"
I20260811 22:09:53.527858 5762 raft_consensus.cc:3037] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:53.527918 5584 raft_consensus.cc:3037] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:53.528905 5584 raft_consensus.cc:2454] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb28c8c5cf384328b2b6ced6b59ac57f in term 1.
I20260811 22:09:53.529044 5762 raft_consensus.cc:2454] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fb28c8c5cf384328b2b6ced6b59ac57f in term 1.
I20260811 22:09:53.529292 5847 leader_election.cc:302] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d1adb8f1e4ae465ba20fe766bdafdd1a, fb28c8c5cf384328b2b6ced6b59ac57f; no voters:
I20260811 22:09:53.529505 6084 raft_consensus.cc:2781] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:53.529665 6084 raft_consensus.cc:686] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 LEADER]: Becoming Leader. State: Replica: fb28c8c5cf384328b2b6ced6b59ac57f, State: Running, Role: LEADER
I20260811 22:09:53.529886 6084 consensus_queue.cc:237] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } }
I20260811 22:09:53.531713 5384 catalog_manager.cc:5495] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f reported cstate change: term changed from 0 to 1, leader changed from <none> to fb28c8c5cf384328b2b6ced6b59ac57f (127.28.47.131). New cstate: current_term: 1 leader_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fb28c8c5cf384328b2b6ced6b59ac57f" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 38313 } health_report { overall_health: HEALTHY } } }
I20260811 22:09:53.562083 6014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:53.558693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:33931 (local address 127.28.47.131:38313)
0811 22:09:53.558897 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:53.558902 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:53.559011 (+ 109us) server_negotiation.cc:407] Connection header received
0811 22:09:53.559170 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:53.559173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:53.559221 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:53.559299 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:09:53.559581 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:53.560101 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:53.560965 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:53.561210 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:53.561323 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:53.561564 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:09:53.561654 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:09:53.561740 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:53.561825 (+ 85us) server_negotiation.cc:299] Negotiation successful
0811 22:09:53.561911 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":22,"server-negotiator.queue_time_us":103}
I20260811 22:09:53.562083 6120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:53.558573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:38313 (local address 127.28.47.131:33931)
0811 22:09:53.558903 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:53.558923 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:53.559070 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:53.559322 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:53.559325 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:53.559333 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0811 22:09:53.559478 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:53.559484 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:53.560225 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:53.560230 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:53.560835 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:53.560851 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:53.561325 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:53.561329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:53.561421 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:53.561446 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0811 22:09:53.561743 (+ 297us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0811 22:09:53.561751 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:09:53.561817 (+ 66us) client_negotiation.cc:240] Negotiation successful
0811 22:09:53.561950 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":257,"mutex_wait_us":45,"thread_start_us":75,"threads_started":1}
I20260811 22:09:53.568642 5584 raft_consensus.cc:1261] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Refusing update from remote peer fb28c8c5cf384328b2b6ced6b59ac57f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:53.569155 5762 raft_consensus.cc:1261] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Refusing update from remote peer fb28c8c5cf384328b2b6ced6b59ac57f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:09:53.569231 6084 consensus_queue.cc:1035] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [LEADER]: Connected to new peer: Peer: permanent_uuid: "d1adb8f1e4ae465ba20fe766bdafdd1a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37385 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:53.569665 6085 consensus_queue.cc:1035] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [LEADER]: Connected to new peer: Peer: permanent_uuid: "09707546d2e64cddacb610a7fcd85e6b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38063 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:09:53.615056 5382 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:53.615217 5382 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:09:53.617624 5382 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f22772c039c340eeb24421c8af2519c9: Sending DeleteTablet for 3 replicas of tablet 54d90f76f24d4836b9dee3886edf27e0
I20260811 22:09:53.618352 5739 tablet_service.cc:1514] Processing DeleteTablet for tablet 54d90f76f24d4836b9dee3886edf27e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:53 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:38544
I20260811 22:09:53.618503 5564 tablet_service.cc:1514] Processing DeleteTablet for tablet 54d90f76f24d4836b9dee3886edf27e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:53 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:37128
I20260811 22:09:53.618538 5905 tablet_service.cc:1514] Processing DeleteTablet for tablet 54d90f76f24d4836b9dee3886edf27e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:09:53 UTC) from {username='oryx', principal='oryx/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:36180
I20260811 22:09:53.618686 6043 tablet_replica.cc:330] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b: stopping tablet replica
I20260811 22:09:53.618750 6045 tablet_replica.cc:330] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a: stopping tablet replica
I20260811 22:09:53.618814 6043 raft_consensus.cc:2229] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:53.618896 6045 raft_consensus.cc:2229] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:09:53.618952 6043 raft_consensus.cc:2258] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:53.618989 6046 tablet_replica.cc:330] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f: stopping tablet replica
I20260811 22:09:53.619033 6045 raft_consensus.cc:2258] T 54d90f76f24d4836b9dee3886edf27e0 P d1adb8f1e4ae465ba20fe766bdafdd1a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:53.619102 6046 raft_consensus.cc:2229] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:09:53.619334 6046 raft_consensus.cc:2258] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:09:53.619617 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 5479
I20260811 22:09:53.619886 6043 ts_tablet_manager.cc:1905] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:53.623497 6043 ts_tablet_manager.cc:1918] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:53.623611 6043 log.cc:1196] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/54d90f76f24d4836b9dee3886edf27e0
I20260811 22:09:53.623986 6043 ts_tablet_manager.cc:1939] T 54d90f76f24d4836b9dee3886edf27e0 P 09707546d2e64cddacb610a7fcd85e6b: Deleting consensus metadata
I20260811 22:09:53.625082 5369 catalog_manager.cc:4841] TS 09707546d2e64cddacb610a7fcd85e6b (127.28.47.130:38063): tablet 54d90f76f24d4836b9dee3886edf27e0 (table test-table [id=b88731c072de461980322c43354ce1c1]) successfully deleted
I20260811 22:09:53.627234 6046 ts_tablet_manager.cc:1905] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:09:53.629415 6046 ts_tablet_manager.cc:1918] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260811 22:09:53.629521 6046 log.cc:1196] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/54d90f76f24d4836b9dee3886edf27e0
I20260811 22:09:53.629779 6046 ts_tablet_manager.cc:1939] T 54d90f76f24d4836b9dee3886edf27e0 P fb28c8c5cf384328b2b6ced6b59ac57f: Deleting consensus metadata
I20260811 22:09:53.630532 5367 catalog_manager.cc:4841] TS fb28c8c5cf384328b2b6ced6b59ac57f (127.28.47.131:38313): tablet 54d90f76f24d4836b9dee3886edf27e0 (table test-table [id=b88731c072de461980322c43354ce1c1]) successfully deleted
W20260811 22:09:53.634958 5367 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:37385 (error 108)
I20260811 22:09:53.635068 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 5660
I20260811 22:09:53.640889 6126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:53.639615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:37385 (local address 127.0.0.1:37140)
0811 22:09:53.640701 (+ 1086us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:53.640788 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:37385: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1005,"thread_start_us":67,"threads_started":1}
W20260811 22:09:53.641120 5367 catalog_manager.cc:4584] TS d1adb8f1e4ae465ba20fe766bdafdd1a (127.28.47.129:37385): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 54d90f76f24d4836b9dee3886edf27e0: Network error: Client connection negotiation failed: client connection to 127.28.47.129:37385: connect: Connection refused (error 111)
I20260811 22:09:53.647122 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 5827
I20260811 22:09:53.659149 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 5349
2026-08-11T22:09:53Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (3967 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:53 dist-test-slave-vxj2 krb5kdc[6130](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:53 dist-test-slave-vxj2 krb5kdc[6130](info): set up 2 sockets
Aug 11 22:09:53 dist-test-slave-vxj2 krb5kdc[6130](info): commencing operation
krb5kdc: starting...
W20260811 22:09:54.822989 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:09:54 dist-test-slave-vxj2 krb5kdc[6130](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:09:54.969611 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:40183
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:42943
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:40183
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:55.095214 6143 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:55.095561 6143 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:55.095607 6143 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:55.099978 6143 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:55.100080 6143 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:55.100100 6143 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:55.100118 6143 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:55.100134 6143 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:55.106556 6143 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:42943
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:40183
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:40183
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6143
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:55.108117 6143 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:55.109620 6143 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:55.119369 6149 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:55.119369 6151 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:55.119604 6143 server_base.cc:1034] running on GCE node
W20260811 22:09:55.119388 6155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:55.120381 6143 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:55.121666 6143 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:55.122869 6143 hybrid_clock.cc:648] HybridClock initialized: now 1786486195122831 us; error 49 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-11T22:09:55Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1465 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:09:55 dist-test-slave-vxj2 krb5kdc[6160](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:09:55 dist-test-slave-vxj2 krb5kdc[6160](info): set up 2 sockets
Aug 11 22:09:55 dist-test-slave-vxj2 krb5kdc[6160](info): commencing operation
krb5kdc: starting...
W20260811 22:09:56.315006 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.162s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:09:56 dist-test-slave-vxj2 krb5kdc[6160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:09:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:09:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:09:56.461365 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45739
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:45739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:56.583114 6175 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:56.583505 6175 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:56.583587 6175 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:56.587971 6175 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:56.588101 6175 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:56.588138 6175 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:56.588182 6175 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:56.588215 6175 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:56.594115 6175 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:45739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6175
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:56.595463 6175 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:56.596601 6175 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:56.605957 6183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:56.605978 6182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:56.605949 6185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:56.606263 6175 server_base.cc:1034] running on GCE node
I20260811 22:09:56.607070 6175 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:56.608345 6175 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:56.609517 6175 hybrid_clock.cc:648] HybridClock initialized: now 1786486196609498 us; error 33 us; skew 500 ppm
Aug 11 22:09:56 dist-test-slave-vxj2 krb5kdc[6160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486196, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:56.616796 6175 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:09:56.617950 6175 webserver.cc:460] Webserver started at http://127.28.47.190:38031/ using document root <none> and password file <none>
I20260811 22:09:56.618539 6175 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:56.618639 6175 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:56.618876 6175 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:56.621095 6175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "bedde9f1aa3843c2ab7c9931c8a60741"
format_stamp: "Formatted at 2026-08-11 22:09:56 on dist-test-slave-vxj2"
server_key: "38700b9caa9bfb1c37bbb6592b9b0a00"
server_key_iv: "cb1ce678758d443c4baafd048801eb80"
server_key_version: "encryptionkey@0"
I20260811 22:09:56.621747 6175 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "bedde9f1aa3843c2ab7c9931c8a60741"
format_stamp: "Formatted at 2026-08-11 22:09:56 on dist-test-slave-vxj2"
server_key: "38700b9caa9bfb1c37bbb6592b9b0a00"
server_key_iv: "cb1ce678758d443c4baafd048801eb80"
server_key_version: "encryptionkey@0"
I20260811 22:09:56.625844 6175 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:09:56.628496 6197 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:56.630126 6175 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:56.630306 6175 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "bedde9f1aa3843c2ab7c9931c8a60741"
format_stamp: "Formatted at 2026-08-11 22:09:56 on dist-test-slave-vxj2"
server_key: "38700b9caa9bfb1c37bbb6592b9b0a00"
server_key_iv: "cb1ce678758d443c4baafd048801eb80"
server_key_version: "encryptionkey@0"
I20260811 22:09:56.630455 6175 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:56.662138 6175 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:56.666508 6175 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:56.666771 6175 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:56.674849 6175 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:45739
I20260811 22:09:56.674868 6259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:45739 every 8 connection(s)
I20260811 22:09:56.676035 6175 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:56.679128 6260 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:09:56.679965 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 6175
I20260811 22:09:56.680073 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:09:56.680428 28862 external_mini_cluster.cc:1428] Setting key 125a21b680b1d1361d919c7301b1202a
I20260811 22:09:56.686736 6260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741: Bootstrap starting.
I20260811 22:09:56.689666 6260 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741: Neither blocks nor log segments found. Creating new log.
I20260811 22:09:56.690551 6260 log.cc:824] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:09:56 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486196, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:56.693269 6260 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741: No bootstrap required, opened a new log
I20260811 22:09:56.697801 6264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:56.682249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49302 (local address 127.28.47.190:45739)
0811 22:09:56.682762 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:56.682781 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:56.682830 (+ 49us) server_negotiation.cc:407] Connection header received
0811 22:09:56.683689 (+ 859us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:56.683714 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:56.684185 (+ 471us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:56.684646 (+ 461us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:56.685683 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:56.687154 (+ 1471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:56.688131 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:56.688821 (+ 690us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:56.688991 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:56.692123 (+ 3132us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:56.692130 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:56.692148 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:56.692179 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:56.694315 (+ 2136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:56.694812 (+ 497us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:56.694818 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:56.694825 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:56.694984 (+ 159us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:56.695268 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:56.695271 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:56.695273 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:56.695893 (+ 620us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:56.696129 (+ 236us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:56.696343 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 22:09:56.696685 (+ 342us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":140,"threads_started":1}
I20260811 22:09:56.698314 6260 raft_consensus.cc:348] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45739 } }
I20260811 22:09:56.698549 6260 raft_consensus.cc:374] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:09:56.698613 6260 raft_consensus.cc:729] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bedde9f1aa3843c2ab7c9931c8a60741, State: Initialized, Role: FOLLOWER
I20260811 22:09:56.699087 6260 consensus_queue.cc:260] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45739 } }
I20260811 22:09:56.699263 6260 raft_consensus.cc:388] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:56.699363 6260 raft_consensus.cc:482] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:56.699493 6260 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:09:56.700672 6260 raft_consensus.cc:504] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45739 } }
I20260811 22:09:56.701058 6260 leader_election.cc:302] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bedde9f1aa3843c2ab7c9931c8a60741; no voters:
I20260811 22:09:56.701445 6260 leader_election.cc:288] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:09:56.701673 6266 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:09:56.701967 6266 raft_consensus.cc:686] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 1 LEADER]: Becoming Leader. State: Replica: bedde9f1aa3843c2ab7c9931c8a60741, State: Running, Role: LEADER
I20260811 22:09:56.702324 6266 consensus_queue.cc:237] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45739 } }
I20260811 22:09:56.702958 6260 sys_catalog.cc:564] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:56.704121 6267 sys_catalog.cc:455] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45739 } } }
I20260811 22:09:56.704264 6267 sys_catalog.cc:458] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:56.704579 6268 sys_catalog.cc:455] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bedde9f1aa3843c2ab7c9931c8a60741. Latest consensus state: current_term: 1 leader_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45739 } } }
I20260811 22:09:56.704666 6277 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:56.704694 6268 sys_catalog.cc:458] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:56.707948 6277 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:56.713979 6277 catalog_manager.cc:1347] Generated new cluster ID: bd9b1b5366b24a85bfe08db4157db657
I20260811 22:09:56.714075 6277 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:56.725382 6277 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:09:56.726878 6277 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:56.735981 6277 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741: Generated new TSK 0
I20260811 22:09:56.737049 6277 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:09:56.772444 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45739
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:56.893617 6297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:56.893998 6297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:56.894048 6297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:56.898360 6297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:56.898452 6297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:56.898515 6297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:56.904692 6297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:56.906113 6297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:56.907287 6297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:56.917759 6303 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:56.917881 6305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:56.917881 6302 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:56.918325 6297 server_base.cc:1034] running on GCE node
I20260811 22:09:56.919795 6297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:56.920585 6297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:56.921805 6297 hybrid_clock.cc:648] HybridClock initialized: now 1786486196921771 us; error 63 us; skew 500 ppm
Aug 11 22:09:56 dist-test-slave-vxj2 krb5kdc[6160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486196, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:56.924893 6297 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:09:56.926013 6297 webserver.cc:460] Webserver started at http://127.28.47.129:37747/ using document root <none> and password file <none>
I20260811 22:09:56.926671 6297 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:56.926779 6297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:56.927021 6297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:56.929332 6297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "edf37cc5e60f440f840bf16c3faaca54"
format_stamp: "Formatted at 2026-08-11 22:09:56 on dist-test-slave-vxj2"
server_key: "93fd951bb99d96da8499107140853f40"
server_key_iv: "f9bbe4e0600d0a5d506afae6dea986d8"
server_key_version: "encryptionkey@0"
I20260811 22:09:56.930001 6297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "edf37cc5e60f440f840bf16c3faaca54"
format_stamp: "Formatted at 2026-08-11 22:09:56 on dist-test-slave-vxj2"
server_key: "93fd951bb99d96da8499107140853f40"
server_key_iv: "f9bbe4e0600d0a5d506afae6dea986d8"
server_key_version: "encryptionkey@0"
I20260811 22:09:56.934510 6297 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:09:56.938437 6315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:56.939978 6297 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:09:56.940167 6297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "edf37cc5e60f440f840bf16c3faaca54"
format_stamp: "Formatted at 2026-08-11 22:09:56 on dist-test-slave-vxj2"
server_key: "93fd951bb99d96da8499107140853f40"
server_key_iv: "f9bbe4e0600d0a5d506afae6dea986d8"
server_key_version: "encryptionkey@0"
I20260811 22:09:56.940320 6297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:56.955734 6297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:56.959937 6297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:56.960328 6297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:56.961145 6297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:56.962258 6297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:56.962342 6297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:56.962427 6297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:56.962478 6297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:56.973867 6297 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:41619
I20260811 22:09:56.973888 6453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:41619 every 8 connection(s)
I20260811 22:09:56.974957 6297 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:56.980561 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 6297
I20260811 22:09:56.980685 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:09:56.981035 28862 external_mini_cluster.cc:1428] Setting key b9d7bf3193b7bcf0aeb33a5b6aaf156a
Aug 11 22:09:56 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486196, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
I20260811 22:09:56.991101 6264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:56.977051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:34453 (local address 127.28.47.190:45739)
0811 22:09:56.977275 (+ 224us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:56.977281 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:56.978227 (+ 946us) server_negotiation.cc:407] Connection header received
0811 22:09:56.979752 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:56.979755 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:56.979819 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:56.979930 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:56.981534 (+ 1604us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:56.982217 (+ 683us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:56.983651 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:56.984149 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:56.984285 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:56.987100 (+ 2815us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:56.987103 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:56.987105 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:56.987134 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:56.988688 (+ 1554us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:56.989352 (+ 664us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:56.989355 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:56.989356 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:56.989442 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:56.989850 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:56.989853 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:56.989854 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:56.990059 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:56.990176 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:56.990756 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 22:09:56.990911 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:09:56.992558 6457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:56.977481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:34453)
0811 22:09:56.978045 (+ 564us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:56.978098 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:56.979495 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:56.980149 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:56.980157 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:56.980597 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:56.981352 (+ 755us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:56.981371 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:56.982432 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:56.982436 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:56.983486 (+ 1050us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:56.983499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:56.984356 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:56.984359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:56.984565 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:56.985290 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:56.985321 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:56.986886 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:56.988839 (+ 1953us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:56.988845 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:56.988865 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:56.989213 (+ 348us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:56.989554 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:56.989558 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:56.989560 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:56.989725 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:56.990176 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:56.990185 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:56.990493 (+ 308us) client_negotiation.cc:769] Sending connection context
0811 22:09:56.990699 (+ 206us) client_negotiation.cc:240] Negotiation successful
0811 22:09:56.991067 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"thread_start_us":117,"threads_started":1}
I20260811 22:09:56.994057 6454 heartbeater.cc:347] Connected to a master server at 127.28.47.190:45739
I20260811 22:09:56.994318 6454 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:09:56.994968 6454 heartbeater.cc:511] Master 127.28.47.190:45739 requested a full tablet report, sending...
I20260811 22:09:56.996773 6218 ts_manager.cc:194] Registered new tserver with Master: edf37cc5e60f440f840bf16c3faaca54 (127.28.47.129:41619)
I20260811 22:09:56.998140 6218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:34453
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:09:57.016958 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45739
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:57.146872 6463 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:57.147192 6463 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:57.147249 6463 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:57.151486 6463 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:57.151590 6463 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:57.151656 6463 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:57.157714 6463 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6463
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:57.158951 6463 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:57.160058 6463 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:57.171239 6469 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:57.171267 6471 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:57.171260 6468 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:57.171667 6463 server_base.cc:1034] running on GCE node
I20260811 22:09:57.172212 6463 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:57.173059 6463 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:57.174316 6463 hybrid_clock.cc:648] HybridClock initialized: now 1786486197174296 us; error 34 us; skew 500 ppm
Aug 11 22:09:57 dist-test-slave-vxj2 krb5kdc[6160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486197, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:57.177728 6463 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:09:57.178833 6463 webserver.cc:460] Webserver started at http://127.28.47.130:33263/ using document root <none> and password file <none>
I20260811 22:09:57.179427 6463 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:57.179492 6463 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:57.179682 6463 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:57.182119 6463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "f167d7998e064f4c922a507fd2fb5026"
format_stamp: "Formatted at 2026-08-11 22:09:57 on dist-test-slave-vxj2"
server_key: "6e922f5da3938f2dba0d8d68b981fe6b"
server_key_iv: "cbc0f1d10163f6e9f59e3c2dd6567540"
server_key_version: "encryptionkey@0"
I20260811 22:09:57.183149 6463 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "f167d7998e064f4c922a507fd2fb5026"
format_stamp: "Formatted at 2026-08-11 22:09:57 on dist-test-slave-vxj2"
server_key: "6e922f5da3938f2dba0d8d68b981fe6b"
server_key_iv: "cbc0f1d10163f6e9f59e3c2dd6567540"
server_key_version: "encryptionkey@0"
I20260811 22:09:57.188169 6463 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:09:57.191200 6479 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:57.192701 6463 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:09:57.192827 6463 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "f167d7998e064f4c922a507fd2fb5026"
format_stamp: "Formatted at 2026-08-11 22:09:57 on dist-test-slave-vxj2"
server_key: "6e922f5da3938f2dba0d8d68b981fe6b"
server_key_iv: "cbc0f1d10163f6e9f59e3c2dd6567540"
server_key_version: "encryptionkey@0"
I20260811 22:09:57.192958 6463 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:57.223444 6463 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:57.227136 6463 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:57.227315 6463 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:57.227928 6463 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:57.228832 6463 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:57.228879 6463 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:57.228924 6463 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:57.228940 6463 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:57.239657 6463 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:37283
I20260811 22:09:57.239688 6618 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:37283 every 8 connection(s)
I20260811 22:09:57.240651 6463 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:57.246613 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 6463
I20260811 22:09:57.246728 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:09:57.247032 28862 external_mini_cluster.cc:1428] Setting key 44b8057789b9a5079027a74293abd441
Aug 11 22:09:57 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486197, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:57.257336 6264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:57.242902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:59209 (local address 127.28.47.190:45739)
0811 22:09:57.243103 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:57.243109 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:57.244095 (+ 986us) server_negotiation.cc:407] Connection header received
0811 22:09:57.245672 (+ 1577us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:57.245676 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:57.245752 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:57.245843 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:57.247649 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:57.248353 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:57.249819 (+ 1466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:57.250069 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:57.250198 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:57.253027 (+ 2829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:57.253031 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:57.253033 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:57.253062 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:57.254898 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:57.255552 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:57.255557 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:57.255559 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:57.255652 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:57.256044 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:57.256048 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:57.256050 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:57.256278 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:57.256397 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:57.256965 (+ 568us) server_negotiation.cc:299] Negotiation successful
0811 22:09:57.257114 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:09:57.258476 6621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:57.243335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:59209)
0811 22:09:57.243906 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:57.243955 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:57.245443 (+ 1488us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:57.246072 (+ 629us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:57.246081 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:57.246718 (+ 637us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:57.247459 (+ 741us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:57.247478 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:57.248559 (+ 1081us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:57.248563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:57.249671 (+ 1108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:57.249683 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:57.250251 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:57.250255 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:57.250464 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:57.251151 (+ 687us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:57.251170 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:57.252794 (+ 1624us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:57.255054 (+ 2260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:57.255061 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:57.255081 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:57.255422 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:57.255765 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:57.255768 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:57.255770 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:57.255936 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:57.256396 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:57.256405 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:57.256701 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 22:09:57.256917 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 22:09:57.257342 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":178,"threads_started":1}
I20260811 22:09:57.260140 6619 heartbeater.cc:347] Connected to a master server at 127.28.47.190:45739
I20260811 22:09:57.260442 6619 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:57.261211 6619 heartbeater.cc:511] Master 127.28.47.190:45739 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
I20260811 22:09:57.262550 6218 ts_manager.cc:194] Registered new tserver with Master: f167d7998e064f4c922a507fd2fb5026 (127.28.47.130:37283)
I20260811 22:09:57.263213 6218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:59209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:09:57.281098 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45739
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:57.411099 6624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:57.411458 6624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:57.411505 6624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:57.416388 6624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:57.416505 6624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:57.416576 6624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:57.422981 6624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:45739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:57.424207 6624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:57.425417 6624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:09:57.438251 6624 server_base.cc:1034] running on GCE node
W20260811 22:09:57.438176 6631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:57.438200 6630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:57.438526 6633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:57.439049 6624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:57.439832 6624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:57.441061 6624 hybrid_clock.cc:648] HybridClock initialized: now 1786486197441025 us; error 47 us; skew 500 ppm
Aug 11 22:09:57 dist-test-slave-vxj2 krb5kdc[6160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486197, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:57.444583 6624 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:09:57.445931 6624 webserver.cc:460] Webserver started at http://127.28.47.131:39475/ using document root <none> and password file <none>
I20260811 22:09:57.446646 6624 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:57.446866 6624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:57.447129 6624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:09:57.449676 6624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "0e32c721a334479eaaff107c79e42eb7"
format_stamp: "Formatted at 2026-08-11 22:09:57 on dist-test-slave-vxj2"
server_key: "c9b1af523fd8c1d41fa5a295dc16dbd9"
server_key_iv: "ada58291d1a415b92930c90f7cb70254"
server_key_version: "encryptionkey@0"
I20260811 22:09:57.450284 6624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "0e32c721a334479eaaff107c79e42eb7"
format_stamp: "Formatted at 2026-08-11 22:09:57 on dist-test-slave-vxj2"
server_key: "c9b1af523fd8c1d41fa5a295dc16dbd9"
server_key_iv: "ada58291d1a415b92930c90f7cb70254"
server_key_version: "encryptionkey@0"
I20260811 22:09:57.454727 6624 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:09:57.458309 6641 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:57.459939 6624 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:09:57.460076 6624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "0e32c721a334479eaaff107c79e42eb7"
format_stamp: "Formatted at 2026-08-11 22:09:57 on dist-test-slave-vxj2"
server_key: "c9b1af523fd8c1d41fa5a295dc16dbd9"
server_key_iv: "ada58291d1a415b92930c90f7cb70254"
server_key_version: "encryptionkey@0"
I20260811 22:09:57.460256 6624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:57.493678 6624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:57.498005 6624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:57.498274 6624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:57.499197 6624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:57.500134 6624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:57.500187 6624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:57.500231 6624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:57.500247 6624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:57.510567 6624 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:42563
I20260811 22:09:57.510587 6781 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:42563 every 8 connection(s)
I20260811 22:09:57.511565 6624 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:57.520721 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 6624
I20260811 22:09:57.520892 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:09:57.521278 28862 external_mini_cluster.cc:1428] Setting key e39b857815f2ebfe358f88bff63cf1f3
Aug 11 22:09:57 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486197, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:09:57.527743 6264 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:57.513768 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:60585 (local address 127.28.47.190:45739)
0811 22:09:57.514042 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:57.514046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:57.514931 (+ 885us) server_negotiation.cc:407] Connection header received
0811 22:09:57.516432 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:57.516436 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:57.516502 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:57.516874 (+ 372us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:57.518522 (+ 1648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:57.519075 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:57.520586 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:57.520810 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:57.520942 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:57.524117 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:57.524121 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:57.524123 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:57.524150 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:57.525578 (+ 1428us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:57.526211 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:57.526214 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:57.526215 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:57.526318 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:57.526671 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:57.526673 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:57.526675 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:57.526901 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:57.526982 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:57.527508 (+ 526us) server_negotiation.cc:299] Negotiation successful
0811 22:09:57.527617 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149}
I20260811 22:09:57.528786 6785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:57.514179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:60585)
0811 22:09:57.514756 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:57.514811 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:57.516174 (+ 1363us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:57.517114 (+ 940us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:57.517123 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:57.517665 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:57.518332 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:57.518349 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:57.519229 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:57.519232 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:57.520375 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:57.520392 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:57.520997 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:57.521002 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:57.521263 (+ 261us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:57.521946 (+ 683us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:57.521963 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:57.523960 (+ 1997us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:09:57.525708 (+ 1748us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:57.525759 (+ 51us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:57.525774 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:57.526105 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:57.526424 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:09:57.526426 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:09:57.526427 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:09:57.526587 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:09:57.527000 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:09:57.527006 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:09:57.527277 (+ 271us) client_negotiation.cc:769] Sending connection context
0811 22:09:57.527460 (+ 183us) client_negotiation.cc:240] Negotiation successful
0811 22:09:57.527754 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":132,"threads_started":1}
I20260811 22:09:57.530246 6783 heartbeater.cc:347] Connected to a master server at 127.28.47.190:45739
I20260811 22:09:57.530540 6783 heartbeater.cc:464] Registering TS with master...
I20260811 22:09:57.531104 6783 heartbeater.cc:511] Master 127.28.47.190:45739 requested a full tablet report, sending...
I20260811 22:09:57.532279 6218 ts_manager.cc:194] Registered new tserver with Master: 0e32c721a334479eaaff107c79e42eb7 (127.28.47.131:42563)
I20260811 22:09:57.532938 6218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:60585
I20260811 22:09:57.535885 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab.
Entry for principal oryx/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab.
WARNING: no policy specified for oryx/127.28.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab.
Entry for principal oryx/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab.
WARNING: no policy specified for oryx/127.28.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.28.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab.
Entry for principal oryx/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab.
I20260811 22:09:57.628554 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 6297
I20260811 22:09:57.639120 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 6463
I20260811 22:09:57.651806 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 6624
I20260811 22:09:57.662990 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 6175
I20260811 22:09:57.674291 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45739
--webserver_interface=127.28.47.190
--webserver_port=38031
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:45739
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:09:57.797820 6795 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:57.798158 6795 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:57.798202 6795 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:57.802556 6795 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:09:57.802637 6795 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:57.802657 6795 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:57.802677 6795 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:09:57.802693 6795 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:09:57.808449 6795 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:45739
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45739
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=38031
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6795
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:57.809808 6795 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:57.810992 6795 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:57.820456 6801 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:57.820513 6804 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:57.820456 6800 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:57.820853 6795 server_base.cc:1034] running on GCE node
I20260811 22:09:57.821616 6795 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:57.822861 6795 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:57.824033 6795 hybrid_clock.cc:648] HybridClock initialized: now 1786486197824000 us; error 46 us; skew 500 ppm
Aug 11 22:09:57 dist-test-slave-vxj2 krb5kdc[6160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486197, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:57.826992 6795 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:09:57.827926 6795 webserver.cc:460] Webserver started at http://127.28.47.190:38031/ using document root <none> and password file <none>
I20260811 22:09:57.828508 6795 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:57.828585 6795 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:57.832770 6795 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:09:57.835390 6812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:57.836760 6795 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260811 22:09:57.836928 6795 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "bedde9f1aa3843c2ab7c9931c8a60741"
format_stamp: "Formatted at 2026-08-11 22:09:56 on dist-test-slave-vxj2"
server_key: "38700b9caa9bfb1c37bbb6592b9b0a00"
server_key_iv: "cb1ce678758d443c4baafd048801eb80"
server_key_version: "encryptionkey@0"
I20260811 22:09:57.837620 6795 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:57.865084 6795 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:57.869179 6795 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:57.869449 6795 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:57.877945 6795 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:45739
I20260811 22:09:57.877998 6876 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:45739 every 8 connection(s)
I20260811 22:09:57.879238 6795 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:09:57.883234 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 6795
I20260811 22:09:57.883792 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:41619
--local_ip_for_outbound_sockets=127.28.47.129
--tserver_master_addrs=127.28.47.190:45739
--webserver_port=37747
--webserver_interface=127.28.47.129
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.47.129
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:09:57.884439 6877 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:09:57.886587 6877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741: Bootstrap starting.
I20260811 22:09:57.911289 6877 log.cc:824] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741: Log is configured to *not* fsync() on all Append() calls
I20260811 22:09:57.919154 6877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:09:57.919616 6877 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741: Bootstrap complete.
I20260811 22:09:57.924253 6877 raft_consensus.cc:348] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45739 } }
I20260811 22:09:57.924695 6877 raft_consensus.cc:729] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bedde9f1aa3843c2ab7c9931c8a60741, State: Initialized, Role: FOLLOWER
I20260811 22:09:57.925309 6877 consensus_queue.cc:260] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45739 } }
I20260811 22:09:57.925439 6877 raft_consensus.cc:388] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:09:57.925534 6877 raft_consensus.cc:482] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:09:57.925643 6877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:09:57.927822 6877 raft_consensus.cc:504] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45739 } }
I20260811 22:09:57.928208 6877 leader_election.cc:302] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bedde9f1aa3843c2ab7c9931c8a60741; no voters:
I20260811 22:09:57.928601 6877 leader_election.cc:288] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:09:57.928704 6881 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:09:57.928947 6881 raft_consensus.cc:686] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [term 2 LEADER]: Becoming Leader. State: Replica: bedde9f1aa3843c2ab7c9931c8a60741, State: Running, Role: LEADER
I20260811 22:09:57.929322 6881 consensus_queue.cc:237] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45739 } }
I20260811 22:09:57.930034 6877 sys_catalog.cc:564] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:09:57.930378 6884 sys_catalog.cc:455] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bedde9f1aa3843c2ab7c9931c8a60741. Latest consensus state: current_term: 2 leader_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45739 } } }
I20260811 22:09:57.930508 6884 sys_catalog.cc:458] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:57.930784 6883 sys_catalog.cc:455] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bedde9f1aa3843c2ab7c9931c8a60741" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45739 } } }
I20260811 22:09:57.930903 6883 sys_catalog.cc:458] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741 [sys.catalog]: This master's current role is: LEADER
I20260811 22:09:57.930912 6887 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:09:57.935565 6887 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:09:57.936385 6887 catalog_manager.cc:1259] Loaded cluster ID: bd9b1b5366b24a85bfe08db4157db657
I20260811 22:09:57.936458 6887 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:09:57.938997 6887 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:09:57.939787 6887 catalog_manager.cc:5879] T 00000000000000000000000000000000 P bedde9f1aa3843c2ab7c9931c8a60741: Loaded TSK: 0
I20260811 22:09:57.941184 6887 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:09:58.022615 6879 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:58.022936 6879 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:58.023012 6879 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:58.027314 6879 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:58.027423 6879 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:58.027523 6879 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:09:58.034165 6879 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.129.keytab
--principal=oryx/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:41619
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=37747
--tserver_master_addrs=127.28.47.190:45739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6879
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:58.035661 6879 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:58.036778 6879 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:58.048700 6907 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:58.048713 6910 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:58.048713 6908 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:58.049335 6879 server_base.cc:1034] running on GCE node
I20260811 22:09:58.049789 6879 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:58.050585 6879 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:58.051776 6879 hybrid_clock.cc:648] HybridClock initialized: now 1786486198051743 us; error 50 us; skew 500 ppm
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486198, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:58.055281 6879 init.cc:377] Logged in from keytab as oryx/127.28.47.129@KRBTEST.COM (short username oryx)
I20260811 22:09:58.056581 6879 webserver.cc:460] Webserver started at http://127.28.47.129:37747/ using document root <none> and password file <none>
I20260811 22:09:58.057437 6879 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:58.057576 6879 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:58.063601 6879 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:09:58.066743 6919 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:58.068464 6879 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:09:58.068621 6879 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "edf37cc5e60f440f840bf16c3faaca54"
format_stamp: "Formatted at 2026-08-11 22:09:56 on dist-test-slave-vxj2"
server_key: "93fd951bb99d96da8499107140853f40"
server_key_iv: "f9bbe4e0600d0a5d506afae6dea986d8"
server_key_version: "encryptionkey@0"
I20260811 22:09:58.069409 6879 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:58.091423 6879 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:58.096273 6879 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:58.096515 6879 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:58.097493 6879 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:58.098537 6879 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:58.098613 6879 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:58.098691 6879 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:58.098734 6879 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:58.109896 6879 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:41619
I20260811 22:09:58.109921 7066 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:41619 every 8 connection(s)
I20260811 22:09:58.111181 6879 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:09:58.113705 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 6879
I20260811 22:09:58.114351 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:37283
--local_ip_for_outbound_sockets=127.28.47.130
--tserver_master_addrs=127.28.47.190:45739
--webserver_port=33263
--webserver_interface=127.28.47.130
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.47.130
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:09:58.127593 7070 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:58.113766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:48395 (local address 127.28.47.190:45739)
0811 22:09:58.114375 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:58.114392 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:58.115169 (+ 777us) server_negotiation.cc:407] Connection header received
0811 22:09:58.116796 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:58.116808 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:58.117105 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:58.117469 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:58.118681 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.120306 (+ 1625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.121739 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.122299 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.122505 (+ 206us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.126856 (+ 4351us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:48395: BlockingRecv error: recv got EOF from 127.28.47.129:48395 (error 108)
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":134,"threads_started":1}
I20260811 22:09:58.127714 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:58.114129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:48395)
0811 22:09:58.114981 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:58.115026 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:58.116350 (+ 1324us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:58.117548 (+ 1198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:58.117557 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:58.118022 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:58.118490 (+ 468us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.118504 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.120488 (+ 1984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.120492 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.121569 (+ 1077us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.121581 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.122493 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.122498 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.122704 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.123435 (+ 731us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:58.123454 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:58.126608 (+ 3154us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":645,"thread_start_us":265,"threads_started":1}
W20260811 22:09:58.128290 7068 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:45739 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.47.190:45739: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:09:58.135334 7071 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:58.128824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:32951)
0811 22:09:58.129044 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:58.129059 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:58.129141 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:58.129474 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:58.129477 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:58.129689 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:58.129851 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.129857 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.131076 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.131079 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.131848 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.131859 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.132473 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.132476 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.132552 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.133201 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:58.133224 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:58.135208 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144}
W20260811 22:09:58.135546 7070 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:58.128975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:32951 (local address 127.28.47.190:45739)
0811 22:09:58.129109 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:58.129113 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:58.129129 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:09:58.129273 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:58.129276 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:58.129333 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:58.129421 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:58.129971 (+ 550us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.130935 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.131964 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.132353 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.132458 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.135424 (+ 2966us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:32951: BlockingRecv error: recv got EOF from 127.28.47.129:32951 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:09:58.142539 7074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:58.135957 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:56321)
0811 22:09:58.136344 (+ 387us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:58.136357 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:58.136449 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:58.136699 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:58.136701 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:58.136867 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:58.136990 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.136996 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.138075 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.138078 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.138762 (+ 684us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.138773 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.139408 (+ 635us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.139411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.139502 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.140047 (+ 545us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:58.140059 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:58.142352 (+ 2293us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":126,"threads_started":1}
W20260811 22:09:58.142629 7070 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:58.136085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:56321 (local address 127.28.47.190:45739)
0811 22:09:58.136213 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:58.136217 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:58.136430 (+ 213us) server_negotiation.cc:407] Connection header received
0811 22:09:58.136528 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:58.136531 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:58.136579 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:58.136672 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:58.137095 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.137941 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.138889 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.139248 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.139374 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.142512 (+ 3138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:56321: BlockingRecv error: recv got EOF from 127.28.47.129:56321 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260811 22:09:58.142879 7068 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:09:58.243316 7073 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:58.243654 7073 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:58.243775 7073 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:58.248306 7073 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:58.248412 7073 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:58.248509 7073 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:09:58.254384 7073 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.130.keytab
--principal=oryx/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:37283
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=33263
--tserver_master_addrs=127.28.47.190:45739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7073
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:58.255822 7073 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:58.257145 7073 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:58.268088 7084 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:58.268056 7087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:58.268056 7085 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:58.268366 7073 server_base.cc:1034] running on GCE node
I20260811 22:09:58.268934 7073 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:58.269737 7073 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:58.271101 7073 hybrid_clock.cc:648] HybridClock initialized: now 1786486198271080 us; error 28 us; skew 500 ppm
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486198, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:58.274408 7073 init.cc:377] Logged in from keytab as oryx/127.28.47.130@KRBTEST.COM (short username oryx)
I20260811 22:09:58.275605 7073 webserver.cc:460] Webserver started at http://127.28.47.130:33263/ using document root <none> and password file <none>
I20260811 22:09:58.276292 7073 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:58.276417 7073 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:58.281032 7073 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:09:58.283903 7095 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:58.285471 7073 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:58.285650 7073 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "f167d7998e064f4c922a507fd2fb5026"
format_stamp: "Formatted at 2026-08-11 22:09:57 on dist-test-slave-vxj2"
server_key: "6e922f5da3938f2dba0d8d68b981fe6b"
server_key_iv: "cbc0f1d10163f6e9f59e3c2dd6567540"
server_key_version: "encryptionkey@0"
I20260811 22:09:58.286317 7073 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:58.309681 7073 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:58.313874 7073 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:58.314169 7073 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:58.314960 7073 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:58.315966 7073 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:58.316049 7073 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:58.316128 7073 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:58.316180 7073 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:58.326545 7073 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:37283
I20260811 22:09:58.326565 7240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:37283 every 8 connection(s)
I20260811 22:09:58.327755 7073 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:09:58.335117 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 7073
I20260811 22:09:58.335772 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:42563
--local_ip_for_outbound_sockets=127.28.47.131
--tserver_master_addrs=127.28.47.190:45739
--webserver_port=39475
--webserver_interface=127.28.47.131
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:37569
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.28.47.131
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:09:58.342861 7070 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:58.330066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:46773 (local address 127.28.47.190:45739)
0811 22:09:58.330222 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:58.330227 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:58.331478 (+ 1251us) server_negotiation.cc:407] Connection header received
0811 22:09:58.333027 (+ 1549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:58.333031 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:58.333095 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:58.333303 (+ 208us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:58.334751 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.335713 (+ 962us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.337334 (+ 1621us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.337661 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.337759 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.342735 (+ 4976us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:46773: BlockingRecv error: recv got EOF from 127.28.47.130:46773 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:09:58.343899 7243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:58.330603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:46773)
0811 22:09:58.331250 (+ 647us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:58.331322 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:58.332778 (+ 1456us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:58.333491 (+ 713us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:58.333500 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:58.333940 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:58.334503 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.334520 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.335891 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.335894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.337091 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.337103 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.337900 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.337906 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.338180 (+ 274us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.338907 (+ 727us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:58.338935 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:58.342517 (+ 3582us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":140,"threads_started":1}
W20260811 22:09:58.344480 7241 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:45739 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.47.190:45739: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:09:58.350942 7243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:58.345001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:57251)
0811 22:09:58.345270 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:58.345288 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:58.345384 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:58.345669 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:58.345673 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:58.345956 (+ 283us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:58.346069 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.346076 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.346858 (+ 782us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.346862 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.347412 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.347420 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.347883 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.347887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.347984 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.348575 (+ 591us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:58.348587 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:58.350784 (+ 2197us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":152}
W20260811 22:09:58.351126 7070 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:58.345191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:57251 (local address 127.28.47.190:45739)
0811 22:09:58.345438 (+ 247us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:58.345444 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:58.345460 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:09:58.345518 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:58.345521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:58.345560 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:58.345650 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:58.346195 (+ 545us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.346740 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.347539 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.347766 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.347881 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.350955 (+ 3074us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:57251: BlockingRecv error: recv got EOF from 127.28.47.130:57251 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:09:58.356942 7243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:58.351730 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:52925)
0811 22:09:58.351870 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:58.351890 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:58.351989 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:58.352270 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:58.352274 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:58.352441 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:58.352519 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.352525 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.353332 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.353335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.353864 (+ 529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.353872 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.354337 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.354340 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.354422 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.354953 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:58.354965 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:58.356801 (+ 1836us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 22:09:58.357242 7241 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:09:58.357380 7070 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:58.351901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:52925 (local address 127.28.47.190:45739)
0811 22:09:58.352070 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:58.352074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:58.352087 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:09:58.352128 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:58.352130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:58.352167 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:58.352254 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:58.352635 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.353202 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.353986 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.354215 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.354333 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.357206 (+ 2873us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:52925: BlockingRecv error: recv got EOF from 127.28.47.130:52925 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
W20260811 22:09:58.462401 7244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:09:58.462761 7244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:09:58.462842 7244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:09:58.467245 7244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:09:58.467360 7244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:09:58.467463 7244 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:09:58.475845 7244 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:37569
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/krb5kdc/oryx_127.28.47.131.keytab
--principal=oryx/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:42563
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=39475
--tserver_master_addrs=127.28.47.190:45739
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:09:58.478142 7244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:09:58.479773 7244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:09:58.490793 7254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:58.490811 7252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:09:58.490828 7251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:09:58.491026 7244 server_base.cc:1034] running on GCE node
I20260811 22:09:58.491855 7244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:09:58.492702 7244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:09:58.493944 7244 hybrid_clock.cc:648] HybridClock initialized: now 1786486198493927 us; error 54 us; skew 500 ppm
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486198, etypes {rep=17 tkt=17 ses=17}, oryx/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:09:58.497319 7244 init.cc:377] Logged in from keytab as oryx/127.28.47.131@KRBTEST.COM (short username oryx)
I20260811 22:09:58.498413 7244 webserver.cc:460] Webserver started at http://127.28.47.131:39475/ using document root <none> and password file <none>
I20260811 22:09:58.499006 7244 fs_manager.cc:359] Metadata directory not provided
I20260811 22:09:58.499092 7244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:09:58.503712 7244 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:09:58.506527 7263 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:58.508074 7244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:09:58.508212 7244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "0e32c721a334479eaaff107c79e42eb7"
format_stamp: "Formatted at 2026-08-11 22:09:57 on dist-test-slave-vxj2"
server_key: "c9b1af523fd8c1d41fa5a295dc16dbd9"
server_key_iv: "ada58291d1a415b92930c90f7cb70254"
server_key_version: "encryptionkey@0"
I20260811 22:09:58.508972 7244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:09:58.527860 7244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:09:58.531912 7244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:09:58.532164 7244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:09:58.532846 7244 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:09:58.533875 7244 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:09:58.533955 7244 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:58.534034 7244 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:09:58.534085 7244 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:09:58.544420 7244 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:42563
I20260811 22:09:58.544512 7405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:42563 every 8 connection(s)
I20260811 22:09:58.545678 7244 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:09:58.555431 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 7244
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:09:58.559072 7070 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:58.548021 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:36023 (local address 127.28.47.190:45739)
0811 22:09:58.548188 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:58.548192 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:58.549306 (+ 1114us) server_negotiation.cc:407] Connection header received
0811 22:09:58.550779 (+ 1473us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:58.550782 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:58.550848 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:58.550966 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:58.552468 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.553009 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.554339 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.554566 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.554682 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.558957 (+ 4275us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:36023: BlockingRecv error: recv got EOF from 127.28.47.131:36023 (error 108)
Metrics: {"mutex_wait_us":127,"server-negotiator.queue_time_us":69}
I20260811 22:09:58.559902 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:58.548469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:36023)
0811 22:09:58.549085 (+ 616us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:58.549130 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:58.550520 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:58.551185 (+ 665us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:58.551193 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:58.551635 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:58.552200 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.552216 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.553197 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.553205 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.554171 (+ 966us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.554182 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.554754 (+ 572us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.554758 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.554950 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.555724 (+ 774us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:58.555747 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:58.558743 (+ 2996us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":247,"threads_started":1}
W20260811 22:09:58.560479 7406 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:45739 (0 consecutive failures): Not authorized: Failed to ping master at 127.28.47.190:45739: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:09:58.567443 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:58.561037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:53281)
0811 22:09:58.561264 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:58.561279 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:58.561362 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:58.561654 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:58.561659 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:58.561862 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:58.561968 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.561974 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.562861 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.562867 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.563593 (+ 726us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.563603 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.564183 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.564190 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.564324 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.564816 (+ 492us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:58.564836 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:58.567329 (+ 2493us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":142}
W20260811 22:09:58.567606 7070 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:58.561295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:53281 (local address 127.28.47.190:45739)
0811 22:09:58.561425 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:58.561428 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:58.561442 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:09:58.561477 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:58.561479 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:58.561519 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:58.561596 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:58.562104 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.562724 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.563739 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.564032 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.564127 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.567490 (+ 3363us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:53281: BlockingRecv error: recv got EOF from 127.28.47.131:53281 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:09:58.569134 7410 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:58.556734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49312 (local address 127.28.47.190:45739)
0811 22:09:58.557030 (+ 296us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:58.557034 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:58.557048 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:09:58.557091 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:58.557094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:58.557182 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:58.557291 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:58.558528 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.559535 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.560694 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.561086 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.561233 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.563370 (+ 2137us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:09:58.563376 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:09:58.563397 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:09:58.563429 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:09:58.566083 (+ 2654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:58.566642 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:58.566651 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:58.566662 (+ 11us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:58.566862 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:09:58.567193 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:09:58.567196 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:09:58.567197 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:09:58.567752 (+ 555us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:09:58.567972 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 22:09:58.568716 (+ 744us) server_negotiation.cc:299] Negotiation successful
0811 22:09:58.568943 (+ 227us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":4,"server-negotiator.queue_time_us":218,"thread_start_us":122,"threads_started":1}
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:58 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:09:58.574461 7408 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:58.568105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:44733)
0811 22:09:58.568209 (+ 104us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:58.568225 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:58.568299 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:58.568632 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:58.568635 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:58.568834 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:58.568933 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.568939 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.569893 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.569896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.570660 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:58.570671 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.571272 (+ 601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.571275 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:58.571368 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.571975 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:58.571987 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:58.574296 (+ 2309us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260811 22:09:58.574553 7070 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:58.568197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:44733 (local address 127.28.47.190:45739)
0811 22:09:58.568377 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:58.568383 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:58.568399 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:09:58.568434 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:58.568437 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:58.568485 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:58.568570 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:58.569075 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.569769 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.570827 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:58.571143 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:58.571261 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:58.574425 (+ 3164us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:44733: BlockingRecv error: recv got EOF from 127.28.47.131:44733 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
W20260811 22:09:58.574899 7406 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 22:09:59 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:59 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:09:59.149544 7411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:59.143587 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:39823)
0811 22:09:59.143879 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:59.143893 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:59.144016 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:59.144271 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:59.144274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:59.144478 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:59.144585 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:59.144591 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:59.145489 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:59.145492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:59.146168 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:59.146180 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:59.146671 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:59.146673 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:59.146751 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:59.147400 (+ 649us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:59.147414 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:59.149386 (+ 1972us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":108,"threads_started":1}
W20260811 22:09:59.149791 7413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:59.143705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:39823 (local address 127.28.47.190:45739)
0811 22:09:59.143990 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:59.143994 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:59.144008 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:09:59.144100 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:59.144102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:59.144147 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:59.144231 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:59.144700 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:59.145365 (+ 665us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:59.146302 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:59.146557 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:59.146644 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:59.149626 (+ 2982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:39823: BlockingRecv error: recv got EOF from 127.28.47.129:39823 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":92,"threads_started":1}
Aug 11 22:09:59 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:59 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:09:59.364904 7414 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:59.358109 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:43213)
0811 22:09:59.358500 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:59.358517 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:59.358684 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:59.359048 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:59.359052 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:59.359279 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:59.359409 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:59.359415 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:59.360218 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:59.360225 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:59.360826 (+ 601us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:59.360836 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:59.361423 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:59.361427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:59.361518 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:59.362224 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:59.362236 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:59.364706 (+ 2470us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":101,"threads_started":1}
W20260811 22:09:59.365006 7413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:59.358281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:43213 (local address 127.28.47.190:45739)
0811 22:09:59.358466 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:59.358471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:59.358614 (+ 143us) server_negotiation.cc:407] Connection header received
0811 22:09:59.358807 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:59.358810 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:59.358912 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:59.359022 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:59.359550 (+ 528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:59.360094 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:59.360961 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:59.361284 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:59.361412 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:59.364870 (+ 3458us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:43213: BlockingRecv error: recv got EOF from 127.28.47.130:43213 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 11 22:09:59 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:09:59 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:09:59.581483 7415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:09:59.575627 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:58197)
0811 22:09:59.575929 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 22:09:59.575947 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:09:59.576071 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:09:59.576411 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:09:59.576412 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:09:59.576605 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:09:59.576753 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:59.576765 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:59.577612 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:59.577614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:59.578213 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:09:59.578223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:09:59.578661 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:09:59.578664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:09:59.578744 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:59.579280 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:09:59.579292 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:09:59.581355 (+ 2063us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":118,"threads_started":1}
W20260811 22:09:59.581722 7413 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:09:59.575749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:58197 (local address 127.28.47.190:45739)
0811 22:09:59.575957 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:09:59.575961 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:09:59.576090 (+ 129us) server_negotiation.cc:407] Connection header received
0811 22:09:59.576148 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:09:59.576183 (+ 35us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:09:59.576254 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:09:59.576345 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:09:59.576906 (+ 561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:59.577498 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:59.578328 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:09:59.578556 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:09:59.578637 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:09:59.581563 (+ 2926us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:58197: BlockingRecv error: recv got EOF from 127.28.47.131:58197 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 11 22:10:00 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:00 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:00.157738 7419 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:00.150670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:41109)
0811 22:10:00.151061 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:00.151082 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:00.151225 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:00.151531 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:00.151533 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:00.151729 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:00.151835 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:00.151840 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:00.152683 (+ 843us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:00.152686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:00.153467 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:00.153482 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:00.154213 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:00.154219 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:00.154366 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:00.154942 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:00.154967 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:00.157581 (+ 2614us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":121,"threads_started":1}
W20260811 22:10:00.157971 7420 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:00.150958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:41109 (local address 127.28.47.190:45739)
0811 22:10:00.151281 (+ 323us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:00.151285 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:00.151300 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:10:00.151343 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:00.151345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:00.151388 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:00.151480 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:00.151982 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:00.152539 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:00.153618 (+ 1079us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:00.154010 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:00.154133 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:00.157755 (+ 3622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:41109: BlockingRecv error: recv got EOF from 127.28.47.129:41109 (error 108)
Metrics: {"server-negotiator.queue_time_us":245,"thread_start_us":110,"threads_started":1}
Aug 11 22:10:00 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:00 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:00.372653 7421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:00.366134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:37303)
0811 22:10:00.366463 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:00.366488 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:00.366627 (+ 139us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:00.367044 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:00.367050 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:00.367327 (+ 277us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:00.367476 (+ 149us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:00.367496 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:00.368474 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:00.368477 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:00.369070 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:00.369080 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:00.369707 (+ 627us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:00.369710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:00.369780 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:00.370360 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:00.370372 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:00.372522 (+ 2150us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":113,"threads_started":1}
W20260811 22:10:00.372864 7420 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:00.366252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:37303 (local address 127.28.47.190:45739)
0811 22:10:00.366461 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:00.366466 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:00.366573 (+ 107us) server_negotiation.cc:407] Connection header received
0811 22:10:00.366726 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:00.366750 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:00.366817 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:00.366908 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:00.367618 (+ 710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:00.368334 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:00.369310 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:00.369573 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:00.369668 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:00.372715 (+ 3047us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:37303: BlockingRecv error: recv got EOF from 127.28.47.130:37303 (error 108)
Metrics: {"server-negotiator.queue_time_us":88}
Aug 11 22:10:00 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:00 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:00.588984 7423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:00.582417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:49373)
0811 22:10:00.582733 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:00.582748 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:00.582844 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:00.583203 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:00.583205 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:00.583429 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:00.583533 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:00.583539 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:00.584355 (+ 816us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:00.584357 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:00.584970 (+ 613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:00.584981 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:00.585486 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:00.585489 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:00.585551 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:00.586133 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:00.586150 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:00.588742 (+ 2592us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":118,"threads_started":1}
W20260811 22:10:00.589021 7420 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:00.582613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:49373 (local address 127.28.47.190:45739)
0811 22:10:00.582773 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:00.582776 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:00.582793 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:10:00.582981 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:00.582983 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:00.583033 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:00.583112 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:00.583667 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:00.584238 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:00.585111 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:00.585376 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:00.585467 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:00.588914 (+ 3447us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:49373: BlockingRecv error: recv got EOF from 127.28.47.131:49373 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 11 22:10:01 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:01 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:01.167871 7424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:01.158888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:52121)
0811 22:10:01.159243 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:01.159258 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:01.159367 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:01.159839 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:01.159842 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:01.160055 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:01.160166 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:01.160172 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:01.161187 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:01.161190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:01.161767 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:01.161778 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:01.162660 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:01.162665 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:01.162786 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:01.163748 (+ 962us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:01.163769 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:01.167598 (+ 3829us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":103,"threads_started":1}
W20260811 22:10:01.167959 7425 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:01.158974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:52121 (local address 127.28.47.190:45739)
0811 22:10:01.159344 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:01.159347 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:01.159364 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:10:01.159552 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:01.159555 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:01.159605 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:01.159699 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:01.160357 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:01.161005 (+ 648us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:01.162030 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:01.162369 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:01.162587 (+ 218us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:01.167807 (+ 5220us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:52121: BlockingRecv error: recv got EOF from 127.28.47.129:52121 (error 108)
Metrics: {"server-negotiator.queue_time_us":291,"thread_start_us":145,"threads_started":1}
Aug 11 22:10:01 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:01 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:01.381306 7426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:01.373796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:41387)
0811 22:10:01.374156 (+ 360us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:01.374175 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:01.374301 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:01.374678 (+ 377us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:01.374680 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:01.374892 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:01.375018 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:01.375024 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:01.375939 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:01.375942 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:01.376584 (+ 642us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:01.376595 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:01.377393 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:01.377398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:01.377476 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:01.378188 (+ 712us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:01.378202 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:01.381059 (+ 2857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":139,"threads_started":1}
W20260811 22:10:01.381466 7425 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:01.373958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:41387 (local address 127.28.47.190:45739)
0811 22:10:01.374101 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:01.374105 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:01.374266 (+ 161us) server_negotiation.cc:407] Connection header received
0811 22:10:01.374451 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:01.374455 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:01.374530 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:01.374628 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:01.375181 (+ 553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:01.375770 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:01.376785 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:01.377190 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:01.377330 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:01.381308 (+ 3978us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:41387: BlockingRecv error: recv got EOF from 127.28.47.130:41387 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 22:10:01 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:01 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:01.597617 7427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:01.590539 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:57019)
0811 22:10:01.590984 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:01.591102 (+ 118us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:01.591382 (+ 280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:01.591745 (+ 363us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:01.591749 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:01.592010 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:01.592123 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:01.592130 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:01.593024 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:01.593030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:01.593704 (+ 674us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:01.593714 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:01.594272 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:01.594276 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:01.594378 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:01.594997 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:01.595010 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:01.597433 (+ 2423us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":336,"thread_start_us":104,"threads_started":1}
W20260811 22:10:01.597764 7425 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:01.590599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:57019 (local address 127.28.47.190:45739)
0811 22:10:01.590982 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:01.590989 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:01.591273 (+ 284us) server_negotiation.cc:407] Connection header received
0811 22:10:01.591517 (+ 244us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:01.591523 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:01.591598 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:01.591722 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:01.592265 (+ 543us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:01.592867 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:01.593849 (+ 982us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:01.594123 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:01.594256 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:01.597603 (+ 3347us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:57019: BlockingRecv error: recv got EOF from 127.28.47.131:57019 (error 108)
Metrics: {"server-negotiator.queue_time_us":194}
Aug 11 22:10:02 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:02 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:02.179972 7428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:02.168973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:60753)
0811 22:10:02.169307 (+ 334us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:02.169321 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:02.169431 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:02.169743 (+ 312us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:02.169746 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:02.169950 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:02.170061 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:02.170067 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:02.170954 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:02.170957 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:02.171596 (+ 639us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:02.171606 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:02.172109 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:02.172113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:02.172189 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:02.177419 (+ 5230us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:02.177439 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:02.179780 (+ 2341us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":99,"threads_started":1}
W20260811 22:10:02.180095 7429 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:02.169044 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:60753 (local address 127.28.47.190:45739)
0811 22:10:02.169330 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:02.169333 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:02.169427 (+ 94us) server_negotiation.cc:407] Connection header received
0811 22:10:02.169512 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:02.169523 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:02.169580 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:02.169671 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:02.170210 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:02.170832 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:02.171734 (+ 902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:02.171982 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:02.172073 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:02.179961 (+ 7888us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:60753: BlockingRecv error: recv got EOF from 127.28.47.129:60753 (error 108)
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":134,"threads_started":1}
Aug 11 22:10:02 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:02 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:02.389384 7430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:02.382736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:43869)
0811 22:10:02.383183 (+ 447us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:02.383211 (+ 28us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:02.383495 (+ 284us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:02.383799 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:02.383802 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:02.384042 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:02.384144 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:02.384151 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:02.384961 (+ 810us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:02.384966 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:02.385652 (+ 686us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:02.385663 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:02.386166 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:02.386170 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:02.386256 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:02.387003 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:02.387016 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:02.389158 (+ 2142us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":101,"threads_started":1}
W20260811 22:10:02.389510 7429 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:02.382929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:43869 (local address 127.28.47.190:45739)
0811 22:10:02.383157 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:02.383162 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:02.383411 (+ 249us) server_negotiation.cc:407] Connection header received
0811 22:10:02.383610 (+ 199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:02.383614 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:02.383674 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:02.383769 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:02.384284 (+ 515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:02.384837 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:02.385794 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:02.386034 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:02.386154 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:02.389388 (+ 3234us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:43869: BlockingRecv error: recv got EOF from 127.28.47.130:43869 (error 108)
Metrics: {"server-negotiator.queue_time_us":111}
Aug 11 22:10:02 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:02 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:10:02.605871 7429 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:02.598885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:52929 (local address 127.28.47.190:45739)
0811 22:10:02.599038 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:02.599041 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:02.599171 (+ 130us) server_negotiation.cc:407] Connection header received
0811 22:10:02.599249 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:02.599253 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:02.599311 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:02.599397 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:02.599896 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:02.600451 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:02.601683 (+ 1232us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:02.602078 (+ 395us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:02.602184 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:02.605696 (+ 3512us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:52929: BlockingRecv error: recv got EOF from 127.28.47.131:52929 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:10:02.605757 7431 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:02.598701 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:52929)
0811 22:10:02.599045 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:02.599058 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:02.599168 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:02.599442 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:02.599444 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:02.599633 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:02.599731 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:02.599736 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:02.600669 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:02.600672 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:02.601512 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:02.601527 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:02.602236 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:02.602240 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:02.602313 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:02.602954 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:02.602967 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:02.605457 (+ 2490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":123,"threads_started":1}
Aug 11 22:10:03 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:03 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:03.186937 7432 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:03.181104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:40165)
0811 22:10:03.181453 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:03.181466 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:03.181552 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:03.181828 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:03.181830 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:03.182041 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:03.182140 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:03.182146 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:03.182963 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:03.182966 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:03.183531 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:03.183540 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:03.184037 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:03.184041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:03.184138 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:03.184841 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:03.184854 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:03.186805 (+ 1951us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":167,"threads_started":1}
W20260811 22:10:03.187114 7433 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:03.181266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:40165 (local address 127.28.47.190:45739)
0811 22:10:03.181518 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:03.181522 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:03.181536 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:10:03.181643 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:03.181645 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:03.181690 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:03.181779 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:03.182251 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:03.182788 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:03.183644 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:03.183865 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:03.183972 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:03.186971 (+ 2999us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:40165: BlockingRecv error: recv got EOF from 127.28.47.129:40165 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":103,"threads_started":1}
Aug 11 22:10:03 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:03 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:03.397521 7434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:03.390485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:52719)
0811 22:10:03.390811 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:03.390831 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:03.390979 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:03.391326 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:03.391328 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:03.391534 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:03.391645 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:03.391651 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:03.392630 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:03.392634 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:03.393580 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:03.393594 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:03.394211 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:03.394216 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:03.394297 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:03.394955 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:03.394981 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:03.397324 (+ 2343us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":98,"threads_started":1}
W20260811 22:10:03.397584 7433 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:03.390837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:52719 (local address 127.28.47.190:45739)
0811 22:10:03.391017 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:03.391026 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:03.391066 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:10:03.391120 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:03.391123 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:03.391184 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:03.391258 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:03.391788 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:03.392451 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:03.393728 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:03.394044 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:03.394148 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:03.397470 (+ 3322us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:52719: BlockingRecv error: recv got EOF from 127.28.47.130:52719 (error 108)
Metrics: {"server-negotiator.queue_time_us":39}
Aug 11 22:10:03 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:03 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:03.615150 7435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:03.607277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:58351)
0811 22:10:03.607713 (+ 436us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:03.607728 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:03.607890 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:03.608210 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:03.608213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:03.608540 (+ 327us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:03.608709 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:03.608716 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:03.609824 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:03.609827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:03.610438 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:03.610448 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:03.611127 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:03.611130 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:03.611208 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:03.612009 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:03.612023 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:03.615008 (+ 2985us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":99,"threads_started":1}
W20260811 22:10:03.615419 7433 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:03.607357 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:58351 (local address 127.28.47.190:45739)
0811 22:10:03.607641 (+ 284us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:03.607647 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:03.607869 (+ 222us) server_negotiation.cc:407] Connection header received
0811 22:10:03.608006 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:03.608009 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:03.608069 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:03.608144 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:03.608991 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:03.609634 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:03.610633 (+ 999us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:03.610876 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:03.611305 (+ 429us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:03.615230 (+ 3925us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:58351: BlockingRecv error: recv got EOF from 127.28.47.131:58351 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
Aug 11 22:10:04 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:04 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:04.194551 7436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:04.188004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:55219)
0811 22:10:04.188386 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:04.188400 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:04.188507 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:04.188881 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:04.188883 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:04.189078 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:04.189209 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:04.189215 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:04.190109 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:04.190113 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:04.190757 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:04.190768 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:04.191356 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:04.191358 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:04.191423 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:04.192092 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:04.192105 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:04.194383 (+ 2278us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":80,"threads_started":1}
W20260811 22:10:04.194798 7437 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:04.188106 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:55219 (local address 127.28.47.190:45739)
0811 22:10:04.188427 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:04.188430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:04.188451 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:10:04.188649 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:04.188652 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:04.188710 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:04.188840 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:04.189326 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:04.189989 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:04.190887 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:04.191222 (+ 335us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:04.191313 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:04.194545 (+ 3232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:55219: BlockingRecv error: recv got EOF from 127.28.47.129:55219 (error 108)
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":135,"threads_started":1}
Aug 11 22:10:04 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:04 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:04.404518 7438 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:04.398478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:47177)
0811 22:10:04.398797 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:04.398812 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:04.398928 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:04.399239 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:04.399241 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:04.399456 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:04.399560 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:04.399566 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:04.400354 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:04.400356 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:04.400909 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:04.400917 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:04.401420 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:04.401423 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:04.401493 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:04.402053 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:04.402065 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:04.404313 (+ 2248us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":91,"threads_started":1}
W20260811 22:10:04.404559 7437 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:04.398677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:47177 (local address 127.28.47.190:45739)
0811 22:10:04.398870 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:04.398873 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:04.398889 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:10:04.399065 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:04.399068 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:04.399117 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:04.399193 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:04.399701 (+ 508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:04.400246 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:04.401033 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:04.401292 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:04.401400 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:04.404456 (+ 3056us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:47177: BlockingRecv error: recv got EOF from 127.28.47.130:47177 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 11 22:10:04 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:04 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:04.622771 7439 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:04.616249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:38941)
0811 22:10:04.616671 (+ 422us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:04.616708 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:04.616844 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:04.617293 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:04.617295 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:04.617535 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:04.617644 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:04.617651 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:04.618591 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:04.618595 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:04.619257 (+ 662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:04.619268 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:04.619781 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:04.619785 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:04.619863 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:04.620536 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:04.620549 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:04.622638 (+ 2089us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":107,"threads_started":1}
W20260811 22:10:04.623067 7437 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:04.616392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:38941 (local address 127.28.47.190:45739)
0811 22:10:04.616539 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:04.616546 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:04.616807 (+ 261us) server_negotiation.cc:407] Connection header received
0811 22:10:04.617003 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:04.617005 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:04.617060 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:04.617139 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:04.617814 (+ 675us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:04.618432 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:04.619404 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:04.619634 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:04.619723 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:04.622931 (+ 3208us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:38941: BlockingRecv error: recv got EOF from 127.28.47.131:38941 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 11 22:10:05 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:05 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:05.202798 7440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:05.195600 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:48141)
0811 22:10:05.196069 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:05.196085 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:05.196199 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:05.196808 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:05.196811 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:05.197047 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:05.197234 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:05.197242 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:05.198161 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:05.198164 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:05.198785 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:05.198795 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:05.199560 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:05.199564 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:05.199645 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:05.200371 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:05.200385 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:05.202657 (+ 2272us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":85,"threads_started":1}
W20260811 22:10:05.203147 7441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:05.195751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:48141 (local address 127.28.47.190:45739)
0811 22:10:05.196308 (+ 557us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:05.196315 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:05.196350 (+ 35us) server_negotiation.cc:407] Connection header received
0811 22:10:05.196429 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:05.196433 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:05.196595 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:05.196737 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:05.197395 (+ 658us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:05.198023 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:05.199006 (+ 983us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:05.199346 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:05.199525 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:05.202997 (+ 3472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:48141: BlockingRecv error: recv got EOF from 127.28.47.129:48141 (error 108)
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":90,"threads_started":1}
Aug 11 22:10:05 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:05 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:05.416479 7443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:05.405610 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:45669)
0811 22:10:05.405956 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:05.405975 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:05.406102 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:05.406601 (+ 499us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:05.406605 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:05.406885 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:05.407010 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:05.407017 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:05.408020 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:05.408025 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:05.408616 (+ 591us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:05.408627 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:05.409187 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:05.409192 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:05.409278 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:05.413923 (+ 4645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:05.413941 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:05.416254 (+ 2313us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":118,"threads_started":1}
W20260811 22:10:05.416589 7441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:05.405787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:45669 (local address 127.28.47.190:45739)
0811 22:10:05.405958 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:05.405961 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:05.406070 (+ 109us) server_negotiation.cc:407] Connection header received
0811 22:10:05.406264 (+ 194us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:05.406267 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:05.406323 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:05.406424 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:05.407157 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:05.407861 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:05.408754 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:05.409028 (+ 274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:05.409155 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:05.416455 (+ 7300us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:45669: BlockingRecv error: recv got EOF from 127.28.47.130:45669 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
Aug 11 22:10:05 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:05 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:05.631887 7445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:05.623777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:37963)
0811 22:10:05.624123 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:05.624143 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:05.624295 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:05.624657 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:05.624660 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:05.624865 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:05.624971 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:05.624976 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:05.626119 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:05.626123 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:05.627329 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:05.627350 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:05.628146 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:05.628153 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:05.628279 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:05.628974 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:05.628991 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:05.631694 (+ 2703us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":89,"threads_started":1}
W20260811 22:10:05.632009 7441 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:05.623887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:37963 (local address 127.28.47.190:45739)
0811 22:10:05.624056 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:05.624060 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:05.624272 (+ 212us) server_negotiation.cc:407] Connection header received
0811 22:10:05.624416 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:05.624420 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:05.624512 (+ 92us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:05.624626 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:05.625109 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:05.625956 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:05.627509 (+ 1553us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:05.627869 (+ 360us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:05.627999 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:05.631841 (+ 3842us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:37963: BlockingRecv error: recv got EOF from 127.28.47.131:37963 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 22:10:06 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:06 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:06.210981 7446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:06.203967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:50931)
0811 22:10:06.204309 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:06.204324 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:06.204446 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:06.204844 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:06.204847 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:06.205056 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:06.205199 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:06.205206 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:06.206062 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:06.206065 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:06.206790 (+ 725us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:06.206802 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:06.207393 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:06.207398 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:06.207505 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:06.208256 (+ 751us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:06.208269 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:06.210832 (+ 2563us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":85,"threads_started":1}
W20260811 22:10:06.211280 7448 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:06.204314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:50931 (local address 127.28.47.190:45739)
0811 22:10:06.204567 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:06.204571 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:06.204585 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:10:06.204627 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:06.204629 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:06.204671 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:06.204758 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:06.205344 (+ 586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:06.205919 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:06.206929 (+ 1010us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:06.207193 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:06.207280 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:06.210983 (+ 3703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:50931: BlockingRecv error: recv got EOF from 127.28.47.129:50931 (error 108)
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":93,"threads_started":1}
Aug 11 22:10:06 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:06 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:06.426805 7449 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:06.417660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:53111)
0811 22:10:06.418022 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:06.418045 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:06.418176 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:06.418631 (+ 455us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:06.418635 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:06.419100 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:06.419663 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:06.419681 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:06.421373 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:06.421377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:06.422323 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:06.422350 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:06.423284 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:06.423287 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:06.423395 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:06.424117 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:06.424130 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:06.426664 (+ 2534us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":90,"threads_started":1}
W20260811 22:10:06.427114 7448 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:06.417808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:53111 (local address 127.28.47.190:45739)
0811 22:10:06.418016 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:06.418020 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:06.418146 (+ 126us) server_negotiation.cc:407] Connection header received
0811 22:10:06.418275 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:06.418278 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:06.418358 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:06.418455 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:06.419987 (+ 1532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:06.421119 (+ 1132us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:06.422801 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:06.423099 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:06.423232 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:06.426978 (+ 3746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:53111: BlockingRecv error: recv got EOF from 127.28.47.130:53111 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 22:10:06 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:06 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:06.638854 7450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:06.632851 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:54653)
0811 22:10:06.633190 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:06.633204 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:06.633351 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:06.633652 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:06.633654 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:06.633866 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:06.633968 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:06.633973 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:06.634762 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:06.634766 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:06.635331 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:06.635339 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:06.635824 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:06.635827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:06.635923 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:06.636437 (+ 514us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:06.636449 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:06.638703 (+ 2254us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":73,"threads_started":1}
W20260811 22:10:06.639195 7448 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:06.632933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:54653 (local address 127.28.47.190:45739)
0811 22:10:06.633139 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:06.633143 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:06.633283 (+ 140us) server_negotiation.cc:407] Connection header received
0811 22:10:06.633476 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:06.633480 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:06.633533 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:06.633623 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:06.634088 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:06.634648 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:06.635466 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:06.635695 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:06.635804 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:06.638889 (+ 3085us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:54653: BlockingRecv error: recv got EOF from 127.28.47.131:54653 (error 108)
Metrics: {"server-negotiator.queue_time_us":121}
Aug 11 22:10:07 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:07 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:07.222800 7451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:07.211960 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:44045)
0811 22:10:07.212452 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:07.212471 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:07.212587 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:07.212924 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:07.212926 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:07.213124 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:07.213293 (+ 169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:07.213300 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:07.214189 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:07.214192 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:07.214848 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:07.214857 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:07.215344 (+ 487us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:07.215346 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:07.215413 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:07.220397 (+ 4984us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:07.220416 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:07.222557 (+ 2141us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":120,"threads_started":1}
W20260811 22:10:07.222903 7452 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:07.212185 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:44045 (local address 127.28.47.190:45739)
0811 22:10:07.212447 (+ 262us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:07.212450 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:07.212607 (+ 157us) server_negotiation.cc:407] Connection header received
0811 22:10:07.212708 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:07.212712 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:07.212779 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:07.212866 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:07.213436 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:07.214079 (+ 643us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:07.214999 (+ 920us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:07.215230 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:07.215319 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:07.222791 (+ 7472us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:44045: BlockingRecv error: recv got EOF from 127.28.47.129:44045 (error 108)
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":70,"threads_started":1}
Aug 11 22:10:07 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:07 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:07.434258 7453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:07.427796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:36363)
0811 22:10:07.428162 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:07.428179 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:07.428313 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:07.428678 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:07.428680 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:07.428886 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:07.428994 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:07.429000 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:07.429930 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:07.429933 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:07.430592 (+ 659us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:07.430602 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:07.431169 (+ 567us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:07.431172 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:07.431239 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:07.431901 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:07.431914 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:07.434130 (+ 2216us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":153,"threads_started":1}
W20260811 22:10:07.434463 7452 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:07.427958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:36363 (local address 127.28.47.190:45739)
0811 22:10:07.428192 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:07.428196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:07.428281 (+ 85us) server_negotiation.cc:407] Connection header received
0811 22:10:07.428454 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:07.428457 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:07.428519 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:07.428595 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:07.429144 (+ 549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:07.429806 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:07.430730 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:07.431064 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:07.431155 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:07.434315 (+ 3160us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:36363: BlockingRecv error: recv got EOF from 127.28.47.130:36363 (error 108)
Metrics: {"server-negotiator.queue_time_us":154}
Aug 11 22:10:07 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:07 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:07.646034 7455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:07.639649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:56897)
0811 22:10:07.639918 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:07.639959 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:07.640063 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:07.640317 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:07.640319 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:07.640559 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:07.640666 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:07.640673 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:07.641582 (+ 909us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:07.641584 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:07.642265 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:07.642276 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:07.642821 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:07.642824 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:07.642900 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:07.643519 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:07.643531 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:07.645887 (+ 2356us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":193,"thread_start_us":77,"threads_started":1}
W20260811 22:10:07.646318 7452 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:07.639753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:56897 (local address 127.28.47.190:45739)
0811 22:10:07.639925 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:07.639930 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:07.640030 (+ 100us) server_negotiation.cc:407] Connection header received
0811 22:10:07.640144 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:07.640146 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:07.640199 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:07.640267 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:07.640804 (+ 537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:07.641409 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:07.642383 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:07.642693 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:07.642795 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:07.646123 (+ 3328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:56897: BlockingRecv error: recv got EOF from 127.28.47.131:56897 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 11 22:10:08 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:08 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:08.230942 7457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:08.224013 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:42213)
0811 22:10:08.224342 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:08.224367 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:08.224508 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:08.224969 (+ 461us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:08.224972 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:08.225280 (+ 308us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:08.225460 (+ 180us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:08.225470 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:08.226568 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:08.226571 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:08.227156 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:08.227165 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:08.227667 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:08.227669 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:08.227736 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:08.228345 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:08.228359 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:08.230801 (+ 2442us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":95,"threads_started":1}
W20260811 22:10:08.231176 7459 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:08.224165 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:42213 (local address 127.28.47.190:45739)
0811 22:10:08.224471 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:08.224482 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:08.224498 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:10:08.224618 (+ 120us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:08.224620 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:08.224704 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:08.224866 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:08.225647 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:08.226414 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:08.227292 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:08.227527 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:08.227616 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:08.231015 (+ 3399us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:42213: BlockingRecv error: recv got EOF from 127.28.47.129:42213 (error 108)
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":102,"threads_started":1}
Aug 11 22:10:08 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:08 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:08.441759 7460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:08.435373 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:35583)
0811 22:10:08.435899 (+ 526us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:08.435914 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:08.436028 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:08.436295 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:08.436297 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:08.436485 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:08.436588 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:08.436594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:08.437497 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:08.437500 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:08.438173 (+ 673us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:08.438185 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:08.438707 (+ 522us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:08.438710 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:08.438778 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:08.439481 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:08.439501 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:08.441631 (+ 2130us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":443,"thread_start_us":137,"threads_started":1}
W20260811 22:10:08.442024 7459 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:08.435581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:35583 (local address 127.28.47.190:45739)
0811 22:10:08.435746 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:08.435750 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:08.435995 (+ 245us) server_negotiation.cc:407] Connection header received
0811 22:10:08.436121 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:08.436123 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:08.436169 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:08.436253 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:08.436706 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:08.437361 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:08.438312 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:08.438589 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:08.438696 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:08.441834 (+ 3138us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:35583: BlockingRecv error: recv got EOF from 127.28.47.130:35583 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 11 22:10:08 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:08 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:08.653950 7461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:08.647212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:40033)
0811 22:10:08.647575 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:08.647589 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:08.647686 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:08.648027 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:08.648030 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:08.648233 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:08.648335 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:08.648341 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:08.649256 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:08.649259 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:08.649826 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:08.649835 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:08.650365 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:08.650369 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:08.650443 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:08.651168 (+ 725us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:08.651181 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:08.653750 (+ 2569us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":133,"threads_started":1}
W20260811 22:10:08.654150 7459 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:08.647475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:40033 (local address 127.28.47.190:45739)
0811 22:10:08.647628 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:08.647631 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:08.647647 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:10:08.647791 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:08.647794 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:08.647849 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:08.647939 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:08.648494 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:08.649065 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:08.649956 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:08.650178 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:08.650301 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:08.654022 (+ 3721us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:40033: BlockingRecv error: recv got EOF from 127.28.47.131:40033 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 11 22:10:09 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:09 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:09.238261 7462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:09.231951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:57391)
0811 22:10:09.232253 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:09.232271 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:09.232414 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:09.232722 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:09.232725 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:09.232969 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:09.233093 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:09.233099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:09.234026 (+ 927us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:09.234031 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:09.234615 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:09.234624 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:09.235155 (+ 531us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:09.235159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:09.235267 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:09.235914 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:09.235927 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:09.238082 (+ 2155us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":95,"threads_started":1}
W20260811 22:10:09.238399 7463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:09.231987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:57391 (local address 127.28.47.190:45739)
0811 22:10:09.232257 (+ 270us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:09.232268 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:09.232360 (+ 92us) server_negotiation.cc:407] Connection header received
0811 22:10:09.232518 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:09.232522 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:09.232584 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:09.232693 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:09.233329 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:09.233899 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:09.234758 (+ 859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:09.235000 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:09.235112 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:09.238261 (+ 3149us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:57391: BlockingRecv error: recv got EOF from 127.28.47.129:57391 (error 108)
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":72,"threads_started":1}
Aug 11 22:10:09 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:09 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:09.450608 7464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:09.442980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:37815)
0811 22:10:09.443263 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:09.443285 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:09.443392 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:09.443754 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:09.443758 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:09.444048 (+ 290us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:09.444178 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:09.444184 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:09.445295 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:09.445298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:09.445927 (+ 629us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:09.445937 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:09.446898 (+ 961us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:09.446901 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:09.446974 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:09.447703 (+ 729us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:09.447724 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:09.450308 (+ 2584us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":88,"threads_started":1}
W20260811 22:10:09.450884 7463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:09.443049 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:37815 (local address 127.28.47.190:45739)
0811 22:10:09.443238 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:09.443243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:09.443364 (+ 121us) server_negotiation.cc:407] Connection header received
0811 22:10:09.443469 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:09.443471 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:09.443533 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:09.443619 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:09.444422 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:09.445075 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:09.446171 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:09.446513 (+ 342us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:09.446932 (+ 419us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:09.450617 (+ 3685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:37815: BlockingRecv error: recv got EOF from 127.28.47.130:37815 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
Aug 11 22:10:09 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:09 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:09.661402 7465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:09.654967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:51737)
0811 22:10:09.655264 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:09.655279 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:09.655393 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:09.655755 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:09.655758 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:09.656006 (+ 248us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:09.656115 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:09.656121 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:09.657106 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:09.657111 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:09.657870 (+ 759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:09.657882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:09.658381 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:09.658384 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:09.658471 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:09.659027 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:09.659039 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:09.661154 (+ 2115us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":108,"threads_started":1}
W20260811 22:10:09.661478 7463 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:09.655143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:51737 (local address 127.28.47.190:45739)
0811 22:10:09.655288 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:09.655292 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:09.655363 (+ 71us) server_negotiation.cc:407] Connection header received
0811 22:10:09.655476 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:09.655479 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:09.655539 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:09.655649 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:09.656262 (+ 613us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:09.656980 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:09.658013 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:09.658257 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:09.658372 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:09.661354 (+ 2982us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:51737: BlockingRecv error: recv got EOF from 127.28.47.131:51737 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 11 22:10:10 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:10 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:10.246519 7467 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:10.239537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:49063)
0811 22:10:10.240109 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:10.240125 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:10.240306 (+ 181us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:10.240636 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:10.240639 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:10.240879 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:10.240989 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:10.240997 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:10.241954 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:10.241959 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:10.242615 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:10.242626 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:10.243149 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:10.243154 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:10.243249 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:10.243883 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:10.243897 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:10.246357 (+ 2460us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":111,"threads_started":1}
W20260811 22:10:10.246778 7468 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:10.239550 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:49063 (local address 127.28.47.190:45739)
0811 22:10:10.239966 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:10.239972 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:10.240233 (+ 261us) server_negotiation.cc:407] Connection header received
0811 22:10:10.240420 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:10.240425 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:10.240488 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:10.240609 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:10.241130 (+ 521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:10.241819 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:10.242757 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:10.243007 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:10.243127 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:10.246615 (+ 3488us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:49063: BlockingRecv error: recv got EOF from 127.28.47.129:49063 (error 108)
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":113,"threads_started":1}
Aug 11 22:10:10 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:10 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:10.458573 7469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:10.452006 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:42005)
0811 22:10:10.452341 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:10.452362 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:10.452519 (+ 157us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:10.452947 (+ 428us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:10.452950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:10.453260 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:10.453386 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:10.453393 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:10.454300 (+ 907us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:10.454303 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:10.454921 (+ 618us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:10.454930 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:10.455424 (+ 494us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:10.455428 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:10.455512 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:10.456141 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:10.456154 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:10.458413 (+ 2259us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":101,"threads_started":1}
W20260811 22:10:10.458775 7468 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:10.452125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:42005 (local address 127.28.47.190:45739)
0811 22:10:10.452312 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:10.452316 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:10.452447 (+ 131us) server_negotiation.cc:407] Connection header received
0811 22:10:10.452727 (+ 280us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:10.452729 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:10.452784 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:10.452855 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:10.453555 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:10.454149 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:10.455063 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:10.455291 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:10.455384 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:10.458601 (+ 3217us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:42005: BlockingRecv error: recv got EOF from 127.28.47.130:42005 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Aug 11 22:10:10 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:10 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:10.668797 7470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:10.662546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:39961)
0811 22:10:10.662913 (+ 367us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:10.662937 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:10.663038 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:10.663280 (+ 242us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:10.663282 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:10.663471 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:10.663566 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:10.663571 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:10.664374 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:10.664376 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:10.664990 (+ 614us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:10.664999 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:10.665697 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:10.665701 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:10.665784 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:10.666519 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:10.666532 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:10.668617 (+ 2085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":91,"threads_started":1}
W20260811 22:10:10.668908 7468 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:10.662624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:39961 (local address 127.28.47.190:45739)
0811 22:10:10.662828 (+ 204us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:10.662839 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:10.663008 (+ 169us) server_negotiation.cc:407] Connection header received
0811 22:10:10.663103 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:10.663105 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:10.663154 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:10.663223 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:10.663678 (+ 455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:10.664259 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:10.665108 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:10.665493 (+ 385us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:10.665581 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:10.668774 (+ 3193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:39961: BlockingRecv error: recv got EOF from 127.28.47.131:39961 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 11 22:10:11 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:11 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:11.254877 7471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:11.247658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:36285)
0811 22:10:11.248020 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:11.248035 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:11.248140 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:11.248484 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:11.248487 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:11.248689 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:11.248796 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:11.248801 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:11.249672 (+ 871us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:11.249675 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:11.250507 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:11.250521 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:11.251212 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:11.251215 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:11.251284 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:11.251915 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:11.251929 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:11.254734 (+ 2805us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":97,"threads_started":1}
W20260811 22:10:11.255146 7475 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:11.247835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:36285 (local address 127.28.47.190:45739)
0811 22:10:11.248083 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:11.248086 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:11.248102 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:10:11.248284 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:11.248287 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:11.248337 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:11.248447 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:11.248903 (+ 456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:11.249559 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:11.250689 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:11.251059 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:11.251145 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:11.255000 (+ 3855us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:36285: BlockingRecv error: recv got EOF from 127.28.47.129:36285 (error 108)
Metrics: {"server-negotiator.queue_time_us":169,"thread_start_us":87,"threads_started":1}
Aug 11 22:10:11 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:11 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:11.466566 7476 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:11.459695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:43735)
0811 22:10:11.460090 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:11.460107 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:11.460243 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:11.460571 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:11.460574 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:11.460805 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:11.460912 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:11.460918 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:11.461774 (+ 856us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:11.461779 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:11.462350 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:11.462358 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:11.462847 (+ 489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:11.462850 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:11.462929 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:11.463552 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:11.463574 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:11.466402 (+ 2828us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":161,"threads_started":1}
W20260811 22:10:11.467082 7475 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:11.459918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:43735 (local address 127.28.47.190:45739)
0811 22:10:11.460092 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:11.460098 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:11.460200 (+ 102us) server_negotiation.cc:407] Connection header received
0811 22:10:11.460395 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:11.460398 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:11.460454 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:11.460554 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:11.461050 (+ 496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:11.461657 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:11.462481 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:11.462713 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:11.462855 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:11.466607 (+ 3752us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:43735: BlockingRecv error: recv got EOF from 127.28.47.130:43735 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 22:10:11 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:11 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:11.677031 7477 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:11.669926 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:37657)
0811 22:10:11.670408 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:11.670426 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:11.670542 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:11.670913 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:11.670916 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:11.671218 (+ 302us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:11.671354 (+ 136us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:11.671380 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:11.672254 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:11.672257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:11.672849 (+ 592us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:11.672860 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:11.673551 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:11.673555 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:11.673639 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:11.674441 (+ 802us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:11.674456 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:11.676832 (+ 2376us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":123,"threads_started":1}
W20260811 22:10:11.677181 7475 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:11.670164 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:37657 (local address 127.28.47.190:45739)
0811 22:10:11.670331 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:11.670337 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:11.670526 (+ 189us) server_negotiation.cc:407] Connection header received
0811 22:10:11.670636 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:11.670641 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:11.670707 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:11.670860 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:11.671498 (+ 638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:11.672096 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:11.673021 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:11.673371 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:11.673478 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:11.677024 (+ 3546us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:37657: BlockingRecv error: recv got EOF from 127.28.47.131:37657 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
Aug 11 22:10:12 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:12 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:12.266896 7478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:12.255825 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:42737)
0811 22:10:12.256204 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:12.256218 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:12.256318 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:12.256712 (+ 394us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:12.256715 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:12.256956 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:12.257065 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:12.257071 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:12.258028 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:12.258032 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:12.258786 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:12.258799 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:12.259335 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:12.259339 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:12.259412 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:12.263959 (+ 4547us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:12.263976 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:12.266746 (+ 2770us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":178,"threads_started":1}
W20260811 22:10:12.267204 7479 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:12.256041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:42737 (local address 127.28.47.190:45739)
0811 22:10:12.256409 (+ 368us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:12.256412 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:12.256427 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:10:12.256471 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:12.256473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:12.256518 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:12.256614 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:12.257241 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:12.257867 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:12.258938 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:12.259189 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:12.259294 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:12.267007 (+ 7713us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:42737: BlockingRecv error: recv got EOF from 127.28.47.129:42737 (error 108)
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":113,"threads_started":1}
Aug 11 22:10:12 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:12 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
W20260811 22:10:12.475589 7479 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:12.467823 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:53207 (local address 127.28.47.190:45739)
0811 22:10:12.468017 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:12.468021 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:12.468344 (+ 323us) server_negotiation.cc:407] Connection header received
0811 22:10:12.468440 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:12.468443 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:12.468527 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:12.468670 (+ 143us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:12.469395 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:12.470016 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:12.471000 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:12.471245 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:12.471342 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:12.475418 (+ 4076us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:53207: BlockingRecv error: recv got EOF from 127.28.47.130:53207 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 22:10:12.475982 7480 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:12.467702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:53207)
0811 22:10:12.468115 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:12.468140 (+ 25us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:12.468285 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:12.468769 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:12.468773 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:12.469003 (+ 230us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:12.469136 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:12.469142 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:12.470191 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:12.470194 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:12.470819 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:12.470828 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:12.471405 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:12.471408 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:12.471483 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:12.472128 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:12.472141 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:12.475820 (+ 3679us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":154,"threads_started":1}
Aug 11 22:10:12 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:12 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.131@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:12.685704 7483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:12.678018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.131:51001)
0811 22:10:12.678397 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:12.678413 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:12.678583 (+ 170us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:12.678945 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:12.678948 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:12.679177 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:12.679278 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:12.679285 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:12.680731 (+ 1446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:12.680735 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:12.681804 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:12.681823 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:12.682353 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:12.682357 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:12.682444 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:12.683099 (+ 655us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:12.683112 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:12.685503 (+ 2391us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":102,"threads_started":1}
W20260811 22:10:12.685832 7479 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:12.678167 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:51001 (local address 127.28.47.190:45739)
0811 22:10:12.678360 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:12.678366 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:12.678512 (+ 146us) server_negotiation.cc:407] Connection header received
0811 22:10:12.678708 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:12.678711 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:12.678774 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:12.678873 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:12.679441 (+ 568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:12.680367 (+ 926us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:12.681963 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:12.682216 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:12.682341 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:12.685685 (+ 3344us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.131:51001: BlockingRecv error: recv got EOF from 127.28.47.131:51001 (error 108)
Metrics: {"server-negotiator.queue_time_us":75}
Aug 11 22:10:13 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:13 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.129@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:13.274122 7484 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:13.267967 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.129:59133)
0811 22:10:13.268268 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:13.268282 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:13.268373 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:13.268713 (+ 340us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:13.268716 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:13.268935 (+ 219us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:13.269044 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:13.269050 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:13.269926 (+ 876us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:13.269929 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:13.270550 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:13.270561 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:13.271071 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:13.271074 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:13.271140 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:13.271717 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:13.271730 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:13.273947 (+ 2217us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":120,"threads_started":1}
W20260811 22:10:13.274339 7485 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:13.268111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:59133 (local address 127.28.47.190:45739)
0811 22:10:13.268387 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:13.268390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:13.268406 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:10:13.268444 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:13.268446 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:13.268495 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:13.268580 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:13.269225 (+ 645us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:13.269794 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:13.270680 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:13.270927 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:13.271020 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:13.274179 (+ 3159us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.129:59133: BlockingRecv error: recv got EOF from 127.28.47.129:59133 (error 108)
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":123,"threads_started":1}
Aug 11 22:10:13 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
Aug 11 22:10:13 dist-test-slave-vxj2 krb5kdc[6160](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.28.47.130@KRBTEST.COM for oryx/127.28.47.190@KRBTEST.COM, Server not found in Kerberos database
I20260811 22:10:13.484222 7487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:13.477327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45739 (local address 127.28.47.130:52123)
0811 22:10:13.477608 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:13.477626 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:13.477753 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:13.478179 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:13.478181 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:13.478384 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:13.478528 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:13.478534 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:13.479539 (+ 1005us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:13.479542 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:13.480239 (+ 697us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:13.480253 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:13.480953 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:13.480956 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:13.481030 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:13.481635 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:13.481650 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:13.484062 (+ 2412us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.28.47.190:45739: Server oryx/127.28.47.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":97,"threads_started":1}
W20260811 22:10:13.484458 7485 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:13.477397 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:52123 (local address 127.28.47.190:45739)
0811 22:10:13.477575 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:13.477584 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:13.477718 (+ 134us) server_negotiation.cc:407] Connection header received
0811 22:10:13.477928 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:13.477931 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:13.478003 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:13.478106 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:13.478681 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:13.479360 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:13.480453 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:13.480741 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:13.480908 (+ 167us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:13.484296 (+ 3388us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.28.47.130:52123: BlockingRecv error: recv got EOF from 127.28.47.130:52123 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:10:13.561002 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 6879
I20260811 22:10:13.570920 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 7073
I20260811 22:10:13.585597 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 7244
I20260811 22:10:13.595830 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 6795
2026-08-11T22:10:13Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18475 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-11T22:10:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:10:13Z Disabled control of system clock
I20260811 22:10:13.642706 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:46601
--webserver_interface=127.28.47.190
--webserver_port=0
--builtin_ntp_servers=127.28.47.148:36185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:46601
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260811 22:10:13.792662 7491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:10:13.793035 7491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:10:13.793077 7491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:10:13.797747 7491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:10:13.797860 7491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:10:13.797878 7491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:10:13.797895 7491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:10:13.797912 7491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:10:13.803732 7491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:36185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:46601
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:46601
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:10:13.805122 7491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:10:13.806910 7491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:10:13.817411 7498 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:13.817406 7501 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:13.817401 7499 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:13.817939 7491 server_base.cc:1034] running on GCE node
I20260811 22:10:13.818926 7491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:10:13.820770 7491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:10:13.822335 7491 hybrid_clock.cc:648] HybridClock initialized: now 1786486213822243 us; error 81 us; skew 500 ppm
I20260811 22:10:13.826349 7491 webserver.cc:460] Webserver started at http://127.28.47.190:37939/ using document root <none> and password file <none>
I20260811 22:10:13.827189 7491 fs_manager.cc:359] Metadata directory not provided
I20260811 22:10:13.827296 7491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:10:13.827585 7491 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:10:13.830435 7491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "7764a2ae7dbe425596b2eb4d623e4c14"
format_stamp: "Formatted at 2026-08-11 22:10:13 on dist-test-slave-vxj2"
server_key: "8991e2026283bfdd59c4c796680f66cc"
server_key_iv: "975be806a7ce309a310f73cd38fdcf89"
server_key_version: "encryptionkey@0"
I20260811 22:10:13.831214 7491 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "7764a2ae7dbe425596b2eb4d623e4c14"
format_stamp: "Formatted at 2026-08-11 22:10:13 on dist-test-slave-vxj2"
server_key: "8991e2026283bfdd59c4c796680f66cc"
server_key_iv: "975be806a7ce309a310f73cd38fdcf89"
server_key_version: "encryptionkey@0"
I20260811 22:10:13.836650 7491 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260811 22:10:13.840070 7508 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:13.842025 7491 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:10:13.842273 7491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "7764a2ae7dbe425596b2eb4d623e4c14"
format_stamp: "Formatted at 2026-08-11 22:10:13 on dist-test-slave-vxj2"
server_key: "8991e2026283bfdd59c4c796680f66cc"
server_key_iv: "975be806a7ce309a310f73cd38fdcf89"
server_key_version: "encryptionkey@0"
I20260811 22:10:13.842454 7491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:10:13.866917 7491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:10:13.867858 7491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:10:13.868129 7491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:10:13.876921 7491 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:46601
I20260811 22:10:13.877017 7575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:46601 every 8 connection(s)
I20260811 22:10:13.878249 7491 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:10:13.882715 7576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:10:13.883701 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 7491
I20260811 22:10:13.883936 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:10:13.884287 28862 external_mini_cluster.cc:1428] Setting key a3bbc82848a995f773eeedbc42254ce6
I20260811 22:10:13.892117 7576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14: Bootstrap starting.
I20260811 22:10:13.895725 7576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14: Neither blocks nor log segments found. Creating new log.
I20260811 22:10:13.897244 7576 log.cc:824] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14: Log is configured to *not* fsync() on all Append() calls
I20260811 22:10:13.897945 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:13.886133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55624 (local address 127.28.47.190:46601)
0811 22:10:13.887067 (+ 934us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:13.887092 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:13.887155 (+ 63us) server_negotiation.cc:407] Connection header received
0811 22:10:13.888244 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:13.888273 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:13.888902 (+ 629us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:13.889394 (+ 492us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:13.889986 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:13.891603 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:13.892880 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:13.893775 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:13.894006 (+ 231us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:13.895040 (+ 1034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:10:13.895048 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:10:13.895069 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:10:13.895690 (+ 621us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:10:13.895785 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:10:13.895831 (+ 46us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:10:13.896168 (+ 337us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:13.896278 (+ 110us) server_negotiation.cc:299] Negotiation successful
0811 22:10:13.896680 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":395,"thread_start_us":150,"threads_started":1}
I20260811 22:10:13.899981 7576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14: No bootstrap required, opened a new log
I20260811 22:10:13.904772 7576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7764a2ae7dbe425596b2eb4d623e4c14" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46601 } }
I20260811 22:10:13.905107 7576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:10:13.905217 7576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7764a2ae7dbe425596b2eb4d623e4c14, State: Initialized, Role: FOLLOWER
I20260811 22:10:13.905802 7576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7764a2ae7dbe425596b2eb4d623e4c14" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46601 } }
I20260811 22:10:13.906101 7576 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:10:13.906797 7576 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:10:13.906947 7576 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:10:13.908499 7576 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7764a2ae7dbe425596b2eb4d623e4c14" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46601 } }
I20260811 22:10:13.909052 7576 leader_election.cc:302] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7764a2ae7dbe425596b2eb4d623e4c14; no voters:
I20260811 22:10:13.909578 7576 leader_election.cc:288] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:10:13.909758 7582 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:10:13.910069 7582 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [term 1 LEADER]: Becoming Leader. State: Replica: 7764a2ae7dbe425596b2eb4d623e4c14, State: Running, Role: LEADER
I20260811 22:10:13.910557 7582 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7764a2ae7dbe425596b2eb4d623e4c14" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46601 } }
I20260811 22:10:13.911206 7576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:10:13.912467 7584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7764a2ae7dbe425596b2eb4d623e4c14. Latest consensus state: current_term: 1 leader_uuid: "7764a2ae7dbe425596b2eb4d623e4c14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7764a2ae7dbe425596b2eb4d623e4c14" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46601 } } }
I20260811 22:10:13.912653 7584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [sys.catalog]: This master's current role is: LEADER
I20260811 22:10:13.912967 7585 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7764a2ae7dbe425596b2eb4d623e4c14" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7764a2ae7dbe425596b2eb4d623e4c14" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46601 } } }
I20260811 22:10:13.913081 7585 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14 [sys.catalog]: This master's current role is: LEADER
I20260811 22:10:13.913112 7592 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:10:13.916823 7592 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:10:13.924993 7592 catalog_manager.cc:1347] Generated new cluster ID: 6c73fb438e6c4fe2afe46ae521e9d147
I20260811 22:10:13.925120 7592 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:10:13.971621 7592 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:10:13.972805 7592 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:10:13.986119 7592 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7764a2ae7dbe425596b2eb4d623e4c14: Generated new TSK 0
I20260811 22:10:13.987265 7592 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:10:13.996690 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46601
--builtin_ntp_servers=127.28.47.148:36185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260811 22:10:14.126271 7604 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:10:14.126636 7604 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:10:14.126678 7604 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:10:14.131181 7604 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:10:14.131270 7604 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:10:14.131333 7604 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:10:14.136646 7604 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:36185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7604
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:10:14.138047 7604 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:10:14.139664 7604 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:10:14.151695 7610 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:14.151719 7616 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:14.151695 7614 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:14.152235 7604 server_base.cc:1034] running on GCE node
I20260811 22:10:14.152753 7604 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:10:14.153563 7604 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:10:14.154798 7604 hybrid_clock.cc:648] HybridClock initialized: now 1786486214154754 us; error 62 us; skew 500 ppm
I20260811 22:10:14.157065 7604 webserver.cc:460] Webserver started at http://127.28.47.129:42813/ using document root <none> and password file <none>
I20260811 22:10:14.157811 7604 fs_manager.cc:359] Metadata directory not provided
I20260811 22:10:14.157972 7604 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:10:14.158231 7604 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:10:14.160633 7604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "bf0662341ac74bba803bf6c5cdd8175f"
format_stamp: "Formatted at 2026-08-11 22:10:14 on dist-test-slave-vxj2"
server_key: "3885d9b738e5c33725a18034bf88c444"
server_key_iv: "571d0c3a669bdf73501ea402821210d7"
server_key_version: "encryptionkey@0"
I20260811 22:10:14.161350 7604 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "bf0662341ac74bba803bf6c5cdd8175f"
format_stamp: "Formatted at 2026-08-11 22:10:14 on dist-test-slave-vxj2"
server_key: "3885d9b738e5c33725a18034bf88c444"
server_key_iv: "571d0c3a669bdf73501ea402821210d7"
server_key_version: "encryptionkey@0"
I20260811 22:10:14.165716 7604 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:10:14.168892 7625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:14.170703 7604 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:10:14.170934 7604 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "bf0662341ac74bba803bf6c5cdd8175f"
format_stamp: "Formatted at 2026-08-11 22:10:14 on dist-test-slave-vxj2"
server_key: "3885d9b738e5c33725a18034bf88c444"
server_key_iv: "571d0c3a669bdf73501ea402821210d7"
server_key_version: "encryptionkey@0"
I20260811 22:10:14.171108 7604 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:10:14.199808 7604 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:10:14.200733 7604 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:10:14.201004 7604 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:10:14.201855 7604 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:10:14.203076 7604 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:10:14.203159 7604 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:14.203241 7604 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:10:14.203289 7604 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:14.214470 7604 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:41351
I20260811 22:10:14.214502 7765 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:41351 every 8 connection(s)
I20260811 22:10:14.215631 7604 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:10:14.216382 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 7604
I20260811 22:10:14.216513 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:10:14.216856 28862 external_mini_cluster.cc:1428] Setting key 12aff39d12cfe91d0f8baa1e95a2ee6e
I20260811 22:10:14.219363 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46601
--builtin_ntp_servers=127.28.47.148:36185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:10:14.230542 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:14.218225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:42583 (local address 127.28.47.190:46601)
0811 22:10:14.218417 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:14.218423 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:14.219996 (+ 1573us) server_negotiation.cc:407] Connection header received
0811 22:10:14.222495 (+ 2499us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:14.222503 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:14.222769 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:14.222966 (+ 197us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:14.224369 (+ 1403us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:14.224950 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:14.227355 (+ 2405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:14.227609 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:14.227722 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:14.229778 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:10:14.229782 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:10:14.229785 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:10:14.229804 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:10:14.229829 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:10:14.229913 (+ 84us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:10:14.230051 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:14.230318 (+ 267us) server_negotiation.cc:299] Negotiation successful
0811 22:10:14.230401 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:10:14.232178 7768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:14.218842 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:46601 (local address 127.28.47.129:42583)
0811 22:10:14.219778 (+ 936us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:14.219845 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:14.222120 (+ 2275us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:14.223141 (+ 1021us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:14.223152 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:14.223260 (+ 108us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:14.224143 (+ 883us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:14.224169 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:14.225117 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:14.225124 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:14.227153 (+ 2029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:14.227176 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:14.227899 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:14.227906 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:14.228229 (+ 323us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:14.229185 (+ 956us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:10:14.229189 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:14.229291 (+ 102us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:10:14.229430 (+ 139us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:10:14.230088 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:10:14.230103 (+ 15us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:10:14.230115 (+ 12us) client_negotiation.cc:769] Sending connection context
0811 22:10:14.230299 (+ 184us) client_negotiation.cc:240] Negotiation successful
0811 22:10:14.230766 (+ 467us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":362,"thread_start_us":141,"threads_started":1}
I20260811 22:10:14.233927 7766 heartbeater.cc:347] Connected to a master server at 127.28.47.190:46601
I20260811 22:10:14.234244 7766 heartbeater.cc:464] Registering TS with master...
I20260811 22:10:14.234943 7766 heartbeater.cc:511] Master 127.28.47.190:46601 requested a full tablet report, sending...
I20260811 22:10:14.236753 7529 ts_manager.cc:194] Registered new tserver with Master: bf0662341ac74bba803bf6c5cdd8175f (127.28.47.129:41351)
I20260811 22:10:14.238092 7529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.47.129:42583
W20260811 22:10:14.350741 7769 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:10:14.351076 7769 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:10:14.351118 7769 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:10:14.355633 7769 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:10:14.355718 7769 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:10:14.355778 7769 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:10:14.361081 7769 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:36185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7769
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:10:14.362533 7769 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:10:14.364158 7769 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:10:14.375926 7775 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:14.375948 7776 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:14.375982 7778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:14.376463 7769 server_base.cc:1034] running on GCE node
I20260811 22:10:14.377028 7769 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:10:14.377864 7769 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:10:14.379065 7769 hybrid_clock.cc:648] HybridClock initialized: now 1786486214379026 us; error 50 us; skew 500 ppm
I20260811 22:10:14.381767 7769 webserver.cc:460] Webserver started at http://127.28.47.130:36693/ using document root <none> and password file <none>
I20260811 22:10:14.382568 7769 fs_manager.cc:359] Metadata directory not provided
I20260811 22:10:14.382699 7769 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:10:14.382982 7769 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:10:14.385607 7769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "b41a9e2142cf4ee5b4bd3e182815a043"
format_stamp: "Formatted at 2026-08-11 22:10:14 on dist-test-slave-vxj2"
server_key: "65be4c1c089eb44a2155003d95653875"
server_key_iv: "6d7029aa4e52b9b656c9189653df6f9b"
server_key_version: "encryptionkey@0"
I20260811 22:10:14.386399 7769 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "b41a9e2142cf4ee5b4bd3e182815a043"
format_stamp: "Formatted at 2026-08-11 22:10:14 on dist-test-slave-vxj2"
server_key: "65be4c1c089eb44a2155003d95653875"
server_key_iv: "6d7029aa4e52b9b656c9189653df6f9b"
server_key_version: "encryptionkey@0"
I20260811 22:10:14.391948 7769 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.005s sys 0.000s
I20260811 22:10:14.395637 7790 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:14.397636 7769 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:10:14.397852 7769 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "b41a9e2142cf4ee5b4bd3e182815a043"
format_stamp: "Formatted at 2026-08-11 22:10:14 on dist-test-slave-vxj2"
server_key: "65be4c1c089eb44a2155003d95653875"
server_key_iv: "6d7029aa4e52b9b656c9189653df6f9b"
server_key_version: "encryptionkey@0"
I20260811 22:10:14.398026 7769 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:10:14.415469 7769 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:10:14.416326 7769 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:10:14.416560 7769 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:10:14.417346 7769 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:10:14.418483 7769 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:10:14.418565 7769 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:14.418643 7769 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:10:14.418687 7769 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:14.429301 7769 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:33595
I20260811 22:10:14.429338 7928 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:33595 every 8 connection(s)
I20260811 22:10:14.430461 7769 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:10:14.440038 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 7769
I20260811 22:10:14.440147 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:10:14.440444 28862 external_mini_cluster.cc:1428] Setting key 4f94663622b49e600b7f2a17bf4f125f
I20260811 22:10:14.440981 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:14.432621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:37873 (local address 127.28.47.190:46601)
0811 22:10:14.432780 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:14.432784 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:14.434081 (+ 1297us) server_negotiation.cc:407] Connection header received
0811 22:10:14.435667 (+ 1586us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:14.435672 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:14.435742 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:14.435864 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:14.436906 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:14.437507 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:14.438767 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:14.438994 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:14.439117 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:14.440446 (+ 1329us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:10:14.440456 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:10:14.440459 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:10:14.440473 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:10:14.440492 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:10:14.440501 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:10:14.440584 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:14.440761 (+ 177us) server_negotiation.cc:299] Negotiation successful
0811 22:10:14.440832 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:10:14.442013 7931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:14.433078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:46601 (local address 127.28.47.130:37873)
0811 22:10:14.433907 (+ 829us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:14.433952 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:14.435396 (+ 1444us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:14.436079 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:14.436087 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:14.436183 (+ 96us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:14.436738 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:14.436752 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:14.437649 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:14.437652 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:14.438614 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:14.438624 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:14.439172 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:14.439176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:14.439367 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:14.440091 (+ 724us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:10:14.440092 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:14.440170 (+ 78us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:10:14.440274 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:10:14.440585 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:10:14.440597 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:10:14.440608 (+ 11us) client_negotiation.cc:769] Sending connection context
0811 22:10:14.440761 (+ 153us) client_negotiation.cc:240] Negotiation successful
0811 22:10:14.441104 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":147,"threads_started":1}
I20260811 22:10:14.442675 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46601
--builtin_ntp_servers=127.28.47.148:36185
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260811 22:10:14.443310 7929 heartbeater.cc:347] Connected to a master server at 127.28.47.190:46601
I20260811 22:10:14.443585 7929 heartbeater.cc:464] Registering TS with master...
I20260811 22:10:14.444185 7929 heartbeater.cc:511] Master 127.28.47.190:46601 requested a full tablet report, sending...
I20260811 22:10:14.445468 7529 ts_manager.cc:194] Registered new tserver with Master: b41a9e2142cf4ee5b4bd3e182815a043 (127.28.47.130:33595)
I20260811 22:10:14.446131 7529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.47.130:37873
W20260811 22:10:14.571327 7932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:10:14.571650 7932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:10:14.571688 7932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:10:14.575973 7932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:10:14.576056 7932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:10:14.576116 7932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:10:14.581523 7932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:36185
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46601
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:10:14.582865 7932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:10:14.584481 7932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:10:14.595778 7939 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:14.595985 7932 server_base.cc:1034] running on GCE node
W20260811 22:10:14.595774 7941 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:14.595819 7944 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:14.596594 7932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:10:14.597455 7932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:10:14.598678 7932 hybrid_clock.cc:648] HybridClock initialized: now 1786486214598654 us; error 37 us; skew 500 ppm
I20260811 22:10:14.600909 7932 webserver.cc:460] Webserver started at http://127.28.47.131:44149/ using document root <none> and password file <none>
I20260811 22:10:14.601688 7932 fs_manager.cc:359] Metadata directory not provided
I20260811 22:10:14.601800 7932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:10:14.602031 7932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:10:14.604700 7932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "c3b5af5ee16a4a8789715babcd6a629a"
format_stamp: "Formatted at 2026-08-11 22:10:14 on dist-test-slave-vxj2"
server_key: "c489744bc2acedd8711a8bd6bf1caac0"
server_key_iv: "0815db51c5a7b1cda543b29c1bc471e5"
server_key_version: "encryptionkey@0"
I20260811 22:10:14.605398 7932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "c3b5af5ee16a4a8789715babcd6a629a"
format_stamp: "Formatted at 2026-08-11 22:10:14 on dist-test-slave-vxj2"
server_key: "c489744bc2acedd8711a8bd6bf1caac0"
server_key_iv: "0815db51c5a7b1cda543b29c1bc471e5"
server_key_version: "encryptionkey@0"
I20260811 22:10:14.609726 7932 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:10:14.613134 7950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:14.614972 7932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:10:14.615171 7932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "c3b5af5ee16a4a8789715babcd6a629a"
format_stamp: "Formatted at 2026-08-11 22:10:14 on dist-test-slave-vxj2"
server_key: "c489744bc2acedd8711a8bd6bf1caac0"
server_key_iv: "0815db51c5a7b1cda543b29c1bc471e5"
server_key_version: "encryptionkey@0"
I20260811 22:10:14.615330 7932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:10:14.625931 7932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:10:14.626713 7932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:10:14.626932 7932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:10:14.627588 7932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:10:14.628685 7932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:10:14.628743 7932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:14.628815 7932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:10:14.628861 7932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:14.639091 7932 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:33997
I20260811 22:10:14.639143 8092 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:33997 every 8 connection(s)
I20260811 22:10:14.640133 7932 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:10:14.642333 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 7932
I20260811 22:10:14.642481 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:10:14.642759 28862 external_mini_cluster.cc:1428] Setting key eea35e61e886c7f25b30a1fc953680ea
I20260811 22:10:14.650290 7579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:14.642305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:35621 (local address 127.28.47.190:46601)
0811 22:10:14.642506 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:14.642512 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:14.643669 (+ 1157us) server_negotiation.cc:407] Connection header received
0811 22:10:14.645146 (+ 1477us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:14.645148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:14.645255 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:14.645324 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:14.646283 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:14.646822 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:14.648132 (+ 1310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:14.648426 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:14.648512 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:14.649829 (+ 1317us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:10:14.649832 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:10:14.649834 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:10:14.649849 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:10:14.649870 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:10:14.649879 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:10:14.649946 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:14.650101 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 22:10:14.650161 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:10:14.651213 8096 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:14.642747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:46601 (local address 127.28.47.131:35621)
0811 22:10:14.643513 (+ 766us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:14.643568 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:14.644953 (+ 1385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:14.645506 (+ 553us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:14.645513 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:14.645607 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:14.646149 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:14.646162 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:14.646959 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:14.646962 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:14.648016 (+ 1054us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:14.648028 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:14.648575 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:14.648578 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:14.648737 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:14.649507 (+ 770us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:10:14.649508 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:14.649555 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:10:14.649631 (+ 76us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:10:14.649962 (+ 331us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:10:14.649970 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:10:14.649975 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:10:14.650079 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 22:10:14.650338 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":160,"threads_started":1}
I20260811 22:10:14.652500 8093 heartbeater.cc:347] Connected to a master server at 127.28.47.190:46601
I20260811 22:10:14.652793 8093 heartbeater.cc:464] Registering TS with master...
I20260811 22:10:14.653491 8093 heartbeater.cc:511] Master 127.28.47.190:46601 requested a full tablet report, sending...
I20260811 22:10:14.655042 7529 ts_manager.cc:194] Registered new tserver with Master: c3b5af5ee16a4a8789715babcd6a629a (127.28.47.131:33997)
I20260811 22:10:14.655758 7529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.47.131:35621
I20260811 22:10:14.656881 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:10:14.660212 7579 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:14.659201 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50476 (local address 127.28.47.190:46601)
0811 22:10:14.659357 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:14.659362 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:14.659574 (+ 212us) server_negotiation.cc:407] Connection header received
0811 22:10:14.659680 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:14.659682 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:14.659724 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:14.659785 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:14.660076 (+ 291us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50476: BlockingRecv error: recv got EOF from 127.0.0.1:50476 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:10:14.660398 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 7604
I20260811 22:10:14.675235 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 7769
I20260811 22:10:14.686859 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 7932
I20260811 22:10:14.696656 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 7491
2026-08-11T22:10:14Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1095 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-11T22:10:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:10:14Z Disabled control of system clock
I20260811 22:10:14.733909 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:43869
--webserver_interface=127.28.47.190
--webserver_port=0
--builtin_ntp_servers=127.28.47.148:41779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:43869
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:10:14.854782 8105 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:10:14.855146 8105 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:10:14.855197 8105 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:10:14.860652 8105 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:10:14.860755 8105 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:10:14.860773 8105 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:10:14.860795 8105 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:10:14.860812 8105 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:10:14.866807 8105 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:41779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:43869
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:43869
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8105
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:10:14.868096 8105 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:10:14.869810 8105 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:10:14.879593 8105 server_base.cc:1034] running on GCE node
W20260811 22:10:14.879535 8113 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:14.879535 8120 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:14.879539 8114 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:14.880474 8105 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:10:14.881865 8105 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:10:14.885502 8105 hybrid_clock.cc:648] HybridClock initialized: now 1786486214885438 us; error 49 us; skew 500 ppm
I20260811 22:10:14.888048 8105 webserver.cc:460] Webserver started at http://127.28.47.190:35395/ using document root <none> and password file <none>
I20260811 22:10:14.889024 8105 fs_manager.cc:359] Metadata directory not provided
I20260811 22:10:14.889163 8105 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:10:14.889477 8105 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:10:14.892009 8105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "e08eca973ef24a0ebde6de83c8cffce3"
format_stamp: "Formatted at 2026-08-11 22:10:14 on dist-test-slave-vxj2"
server_key: "34c46c2a7519452d8564a3eff972f384"
server_key_iv: "9fbc2a7a1a5fd0995ddbfc17b77477a6"
server_key_version: "encryptionkey@0"
I20260811 22:10:14.892697 8105 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "e08eca973ef24a0ebde6de83c8cffce3"
format_stamp: "Formatted at 2026-08-11 22:10:14 on dist-test-slave-vxj2"
server_key: "34c46c2a7519452d8564a3eff972f384"
server_key_iv: "9fbc2a7a1a5fd0995ddbfc17b77477a6"
server_key_version: "encryptionkey@0"
I20260811 22:10:14.897126 8105 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.002s
I20260811 22:10:14.900362 8128 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:14.902100 8105 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:10:14.902324 8105 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "e08eca973ef24a0ebde6de83c8cffce3"
format_stamp: "Formatted at 2026-08-11 22:10:14 on dist-test-slave-vxj2"
server_key: "34c46c2a7519452d8564a3eff972f384"
server_key_iv: "9fbc2a7a1a5fd0995ddbfc17b77477a6"
server_key_version: "encryptionkey@0"
I20260811 22:10:14.902491 8105 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:10:14.908042 8105 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:10:14.908795 8105 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:10:14.909025 8105 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:10:14.917315 8191 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:43869 every 8 connection(s)
I20260811 22:10:14.917321 8105 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:43869
I20260811 22:10:14.918720 8105 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:10:14.921518 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 8105
I20260811 22:10:14.921665 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:10:14.921983 28862 external_mini_cluster.cc:1428] Setting key 1eee46005f336f07af4e89c5d358d9ae
I20260811 22:10:14.923076 8192 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:10:14.929596 8197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:14.923749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50254 (local address 127.28.47.190:43869)
0811 22:10:14.924756 (+ 1007us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:14.924781 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:14.924832 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:10:14.925823 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:14.925851 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:14.926396 (+ 545us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:14.926850 (+ 454us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:14.927849 (+ 999us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:10:14.927855 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:10:14.927872 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:10:14.927896 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:10:14.927973 (+ 77us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:10:14.927998 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:10:14.928157 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:14.928231 (+ 74us) server_negotiation.cc:299] Negotiation successful
0811 22:10:14.928519 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":148,"threads_started":1}
I20260811 22:10:14.931656 8192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3: Bootstrap starting.
I20260811 22:10:14.934468 8192 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3: Neither blocks nor log segments found. Creating new log.
I20260811 22:10:14.935367 8192 log.cc:824] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3: Log is configured to *not* fsync() on all Append() calls
I20260811 22:10:14.938377 8192 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3: No bootstrap required, opened a new log
I20260811 22:10:14.943070 8192 raft_consensus.cc:348] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08eca973ef24a0ebde6de83c8cffce3" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43869 } }
I20260811 22:10:14.943668 8192 raft_consensus.cc:374] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:10:14.943773 8192 raft_consensus.cc:729] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e08eca973ef24a0ebde6de83c8cffce3, State: Initialized, Role: FOLLOWER
I20260811 22:10:14.944329 8192 consensus_queue.cc:260] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08eca973ef24a0ebde6de83c8cffce3" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43869 } }
I20260811 22:10:14.944481 8192 raft_consensus.cc:388] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:10:14.944715 8192 raft_consensus.cc:482] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:10:14.944908 8192 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:10:14.946185 8192 raft_consensus.cc:504] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08eca973ef24a0ebde6de83c8cffce3" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43869 } }
I20260811 22:10:14.946877 8192 leader_election.cc:302] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e08eca973ef24a0ebde6de83c8cffce3; no voters:
I20260811 22:10:14.947320 8192 leader_election.cc:288] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:10:14.947435 8199 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:10:14.947711 8199 raft_consensus.cc:686] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [term 1 LEADER]: Becoming Leader. State: Replica: e08eca973ef24a0ebde6de83c8cffce3, State: Running, Role: LEADER
I20260811 22:10:14.948191 8199 consensus_queue.cc:237] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08eca973ef24a0ebde6de83c8cffce3" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43869 } }
I20260811 22:10:14.949054 8192 sys_catalog.cc:564] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:10:14.950263 8200 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e08eca973ef24a0ebde6de83c8cffce3. Latest consensus state: current_term: 1 leader_uuid: "e08eca973ef24a0ebde6de83c8cffce3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08eca973ef24a0ebde6de83c8cffce3" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43869 } } }
I20260811 22:10:14.950421 8200 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:10:14.950717 8201 sys_catalog.cc:455] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e08eca973ef24a0ebde6de83c8cffce3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e08eca973ef24a0ebde6de83c8cffce3" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43869 } } }
I20260811 22:10:14.950858 8208 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:10:14.950893 8201 sys_catalog.cc:458] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3 [sys.catalog]: This master's current role is: LEADER
I20260811 22:10:14.954815 8208 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:10:14.962668 8208 catalog_manager.cc:1347] Generated new cluster ID: fdb024ac601a4cf393dc69668abd8d80
I20260811 22:10:14.962777 8208 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:10:14.985643 8208 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:10:14.986421 8208 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:10:14.995048 8208 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e08eca973ef24a0ebde6de83c8cffce3: Generated new TSK 0
I20260811 22:10:14.995952 8208 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:10:15.002017 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:43869
--builtin_ntp_servers=127.28.47.148:41779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260811 22:10:15.137498 8219 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:10:15.137840 8219 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:10:15.137884 8219 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:10:15.142469 8219 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:10:15.142552 8219 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:10:15.142616 8219 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:10:15.147931 8219 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:41779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:43869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8219
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:10:15.149402 8219 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:10:15.151453 8219 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:10:15.164862 8225 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:15.164862 8228 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:15.164863 8224 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:15.165375 8219 server_base.cc:1034] running on GCE node
I20260811 22:10:15.165947 8219 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:10:15.166747 8219 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:10:15.168154 8219 hybrid_clock.cc:648] HybridClock initialized: now 1786486215168109 us; error 56 us; skew 500 ppm
I20260811 22:10:15.170610 8219 webserver.cc:460] Webserver started at http://127.28.47.129:34865/ using document root <none> and password file <none>
I20260811 22:10:15.171314 8219 fs_manager.cc:359] Metadata directory not provided
I20260811 22:10:15.171434 8219 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:10:15.171701 8219 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:10:15.174376 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "da8a6f9318334d54909025cdb81d35d5"
format_stamp: "Formatted at 2026-08-11 22:10:15 on dist-test-slave-vxj2"
server_key: "dbd425dbadf6178eac17e2aebddf9aaf"
server_key_iv: "e2455d3270c0d3b84797c189679090fc"
server_key_version: "encryptionkey@0"
I20260811 22:10:15.175084 8219 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "da8a6f9318334d54909025cdb81d35d5"
format_stamp: "Formatted at 2026-08-11 22:10:15 on dist-test-slave-vxj2"
server_key: "dbd425dbadf6178eac17e2aebddf9aaf"
server_key_iv: "e2455d3270c0d3b84797c189679090fc"
server_key_version: "encryptionkey@0"
I20260811 22:10:15.179668 8219 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:10:15.183561 8235 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:15.185143 8219 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.001s
I20260811 22:10:15.185397 8219 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "da8a6f9318334d54909025cdb81d35d5"
format_stamp: "Formatted at 2026-08-11 22:10:15 on dist-test-slave-vxj2"
server_key: "dbd425dbadf6178eac17e2aebddf9aaf"
server_key_iv: "e2455d3270c0d3b84797c189679090fc"
server_key_version: "encryptionkey@0"
I20260811 22:10:15.185559 8219 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:10:15.190811 8219 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:10:15.191577 8219 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:10:15.191794 8219 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:10:15.192454 8219 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:10:15.193596 8219 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:10:15.193672 8219 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:15.193765 8219 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:10:15.193809 8219 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:15.204103 8219 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:44755
I20260811 22:10:15.204123 8375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:44755 every 8 connection(s)
I20260811 22:10:15.205302 8219 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:10:15.211774 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 8219
I20260811 22:10:15.211902 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:10:15.212283 28862 external_mini_cluster.cc:1428] Setting key f1fe0ff187dc3da4863dc88497f5b085
I20260811 22:10:15.213387 8197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:15.207880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:36347 (local address 127.28.47.190:43869)
0811 22:10:15.208029 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:15.208033 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:15.209362 (+ 1329us) server_negotiation.cc:407] Connection header received
0811 22:10:15.210977 (+ 1615us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:15.210980 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:15.211041 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:15.211150 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:15.212669 (+ 1519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:10:15.212672 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:10:15.212674 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:10:15.212689 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:10:15.212710 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:10:15.212718 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:10:15.212857 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:15.213099 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 22:10:15.213182 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:10:15.214648 8379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:15.208232 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:43869 (local address 127.28.47.129:36347)
0811 22:10:15.209123 (+ 891us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:15.209245 (+ 122us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:15.210658 (+ 1413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:15.211362 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:15.211371 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:15.211535 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:15.212277 (+ 742us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:10:15.212279 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:15.212341 (+ 62us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:10:15.212448 (+ 107us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:10:15.212903 (+ 455us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:10:15.212912 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:10:15.212918 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:10:15.213190 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 22:10:15.213506 (+ 316us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":125,"threads_started":1}
I20260811 22:10:15.215152 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:43869
--builtin_ntp_servers=127.28.47.148:41779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 22:10:15.216449 8377 heartbeater.cc:347] Connected to a master server at 127.28.47.190:43869
I20260811 22:10:15.216755 8377 heartbeater.cc:464] Registering TS with master...
I20260811 22:10:15.217374 8377 heartbeater.cc:511] Master 127.28.47.190:43869 requested a full tablet report, sending...
I20260811 22:10:15.219547 8150 ts_manager.cc:194] Registered new tserver with Master: da8a6f9318334d54909025cdb81d35d5 (127.28.47.129:44755)
W20260811 22:10:15.348356 8380 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:10:15.348704 8380 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:10:15.348749 8380 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:10:15.353611 8380 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:10:15.353691 8380 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:10:15.353756 8380 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:10:15.359834 8380 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:41779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:43869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8380
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:10:15.361070 8380 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:10:15.362746 8380 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:10:15.374128 8386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:15.374178 8389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:15.374166 8387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:15.374639 8380 server_base.cc:1034] running on GCE node
I20260811 22:10:15.375208 8380 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:10:15.376057 8380 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:10:15.377267 8380 hybrid_clock.cc:648] HybridClock initialized: now 1786486215377243 us; error 51 us; skew 500 ppm
I20260811 22:10:15.379662 8380 webserver.cc:460] Webserver started at http://127.28.47.130:44071/ using document root <none> and password file <none>
I20260811 22:10:15.380666 8380 fs_manager.cc:359] Metadata directory not provided
I20260811 22:10:15.380805 8380 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:10:15.381126 8380 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:10:15.385629 8380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "2473368e2dfe485da07481b2cfb89dfd"
format_stamp: "Formatted at 2026-08-11 22:10:15 on dist-test-slave-vxj2"
server_key: "24fae6fbf3b3c7aa8cfe69fa7d8b9e75"
server_key_iv: "420533f6889b4a47ad756a02b3afb1b3"
server_key_version: "encryptionkey@0"
I20260811 22:10:15.386639 8380 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "2473368e2dfe485da07481b2cfb89dfd"
format_stamp: "Formatted at 2026-08-11 22:10:15 on dist-test-slave-vxj2"
server_key: "24fae6fbf3b3c7aa8cfe69fa7d8b9e75"
server_key_iv: "420533f6889b4a47ad756a02b3afb1b3"
server_key_version: "encryptionkey@0"
I20260811 22:10:15.398305 8380 fs_manager.cc:691] Time spent creating directory manager: real 0.011s user 0.005s sys 0.000s
I20260811 22:10:15.402843 8395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:15.404627 8380 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:10:15.404786 8380 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "2473368e2dfe485da07481b2cfb89dfd"
format_stamp: "Formatted at 2026-08-11 22:10:15 on dist-test-slave-vxj2"
server_key: "24fae6fbf3b3c7aa8cfe69fa7d8b9e75"
server_key_iv: "420533f6889b4a47ad756a02b3afb1b3"
server_key_version: "encryptionkey@0"
I20260811 22:10:15.405035 8380 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:10:15.410622 8380 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:10:15.411494 8380 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:10:15.411737 8380 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:10:15.412549 8380 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:10:15.413769 8380 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:10:15.413861 8380 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:15.413942 8380 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:10:15.413998 8380 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:15.424882 8380 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:42227
I20260811 22:10:15.424928 8544 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:42227 every 8 connection(s)
I20260811 22:10:15.426071 8380 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:10:15.433806 8197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:15.428420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:57363 (local address 127.28.47.190:43869)
0811 22:10:15.428660 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:15.428667 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:15.430068 (+ 1401us) server_negotiation.cc:407] Connection header received
0811 22:10:15.431584 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:15.431588 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:15.431641 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:15.431725 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:15.433217 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:10:15.433221 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:10:15.433224 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:10:15.433249 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:10:15.433273 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:10:15.433283 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:10:15.433447 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:15.433597 (+ 150us) server_negotiation.cc:299] Negotiation successful
0811 22:10:15.433653 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111}
I20260811 22:10:15.434914 8547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:15.428914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:43869 (local address 127.28.47.130:57363)
0811 22:10:15.429858 (+ 944us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:15.429909 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:15.431308 (+ 1399us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:15.431925 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:15.431933 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:15.432047 (+ 114us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:15.432793 (+ 746us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:10:15.432795 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:15.432846 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:10:15.432970 (+ 124us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:10:15.433422 (+ 452us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:10:15.433433 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:10:15.433440 (+ 7us) client_negotiation.cc:769] Sending connection context
0811 22:10:15.433661 (+ 221us) client_negotiation.cc:240] Negotiation successful
0811 22:10:15.433988 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"thread_start_us":230,"threads_started":1}
I20260811 22:10:15.435931 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 8380
I20260811 22:10:15.436084 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:10:15.436384 28862 external_mini_cluster.cc:1428] Setting key 0ed0ccd1d999ed80a6d443d057a1b45f
I20260811 22:10:15.436570 8545 heartbeater.cc:347] Connected to a master server at 127.28.47.190:43869
I20260811 22:10:15.436897 8545 heartbeater.cc:464] Registering TS with master...
I20260811 22:10:15.437518 8545 heartbeater.cc:511] Master 127.28.47.190:43869 requested a full tablet report, sending...
I20260811 22:10:15.438853 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:43869
--builtin_ntp_servers=127.28.47.148:41779
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260811 22:10:15.438843 8150 ts_manager.cc:194] Registered new tserver with Master: 2473368e2dfe485da07481b2cfb89dfd (127.28.47.130:42227)
W20260811 22:10:15.570739 8548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:10:15.571056 8548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:10:15.571101 8548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:10:15.575795 8548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:10:15.575917 8548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:10:15.575984 8548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:10:15.581796 8548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:41779
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:43869
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:10:15.583261 8548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:10:15.584872 8548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:10:15.597185 8553 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:15.597213 8554 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:15.597185 8556 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:15.597697 8548 server_base.cc:1034] running on GCE node
I20260811 22:10:15.598673 8548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:10:15.599666 8548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:10:15.600948 8548 hybrid_clock.cc:648] HybridClock initialized: now 1786486215600928 us; error 38 us; skew 500 ppm
I20260811 22:10:15.603462 8548 webserver.cc:460] Webserver started at http://127.28.47.131:40681/ using document root <none> and password file <none>
I20260811 22:10:15.604259 8548 fs_manager.cc:359] Metadata directory not provided
I20260811 22:10:15.604385 8548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:10:15.604662 8548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:10:15.607329 8548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "7b0ee3671ce74f71936a7c6f6bbc4158"
format_stamp: "Formatted at 2026-08-11 22:10:15 on dist-test-slave-vxj2"
server_key: "5cb32c81dc47c8dd04db9d335b2c08d6"
server_key_iv: "24445647062aaaba6cf618857268e95a"
server_key_version: "encryptionkey@0"
I20260811 22:10:15.608110 8548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "7b0ee3671ce74f71936a7c6f6bbc4158"
format_stamp: "Formatted at 2026-08-11 22:10:15 on dist-test-slave-vxj2"
server_key: "5cb32c81dc47c8dd04db9d335b2c08d6"
server_key_iv: "24445647062aaaba6cf618857268e95a"
server_key_version: "encryptionkey@0"
I20260811 22:10:15.613291 8548 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.006s sys 0.001s
I20260811 22:10:15.616833 8562 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:15.618620 8548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:10:15.618896 8548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "7b0ee3671ce74f71936a7c6f6bbc4158"
format_stamp: "Formatted at 2026-08-11 22:10:15 on dist-test-slave-vxj2"
server_key: "5cb32c81dc47c8dd04db9d335b2c08d6"
server_key_iv: "24445647062aaaba6cf618857268e95a"
server_key_version: "encryptionkey@0"
I20260811 22:10:15.619068 8548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:10:15.624738 8548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:10:15.625640 8548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:10:15.625895 8548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:10:15.626984 8548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:10:15.628566 8548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:10:15.628684 8548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:15.628772 8548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:10:15.628827 8548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:15.640029 8548 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:42191
I20260811 22:10:15.640110 8703 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:42191 every 8 connection(s)
I20260811 22:10:15.641259 8548 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:10:15.648433 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 8548
I20260811 22:10:15.648586 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:10:15.648965 28862 external_mini_cluster.cc:1428] Setting key 769906abf66de2f72ef1b719710622fc
I20260811 22:10:15.649547 8197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:15.643517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:37195 (local address 127.28.47.190:43869)
0811 22:10:15.643691 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:15.643696 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:15.645108 (+ 1412us) server_negotiation.cc:407] Connection header received
0811 22:10:15.647033 (+ 1925us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:15.647036 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:15.647089 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:15.647177 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:15.648820 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:10:15.648825 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:10:15.648828 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:10:15.648856 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:10:15.648885 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:10:15.648898 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:10:15.649023 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:15.649340 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 22:10:15.649394 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:10:15.650941 8706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:15.644043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:43869 (local address 127.28.47.131:37195)
0811 22:10:15.644885 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:15.644938 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:15.646763 (+ 1825us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:15.647421 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:15.647430 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:15.647543 (+ 113us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:15.648331 (+ 788us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:10:15.648333 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:15.648383 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:10:15.648471 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:10:15.649123 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:10:15.649137 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:10:15.649167 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:10:15.649359 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 22:10:15.649679 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"thread_start_us":157,"threads_started":1}
I20260811 22:10:15.652455 8704 heartbeater.cc:347] Connected to a master server at 127.28.47.190:43869
I20260811 22:10:15.652781 8704 heartbeater.cc:464] Registering TS with master...
I20260811 22:10:15.653378 8704 heartbeater.cc:511] Master 127.28.47.190:43869 requested a full tablet report, sending...
I20260811 22:10:15.654666 8150 ts_manager.cc:194] Registered new tserver with Master: 7b0ee3671ce74f71936a7c6f6bbc4158 (127.28.47.131:42191)
I20260811 22:10:15.665268 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260811 22:10:15.670109 8197 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:10:15.668986 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50260 (local address 127.28.47.190:43869)
0811 22:10:15.669182 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:15.669189 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:15.669213 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:10:15.669289 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:15.669294 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:15.669340 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:15.669448 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:15.669913 (+ 465us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:50260: BlockingRecv error: recv got EOF from 127.0.0.1:50260 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:10:15.670671 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 8219
I20260811 22:10:15.684461 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 8380
I20260811 22:10:15.703629 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 8548
I20260811 22:10:15.735800 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 8105
2026-08-11T22:10:15Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (1037 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:10:15 dist-test-slave-vxj2 krb5kdc[8715](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:10:15 dist-test-slave-vxj2 krb5kdc[8715](info): set up 2 sockets
Aug 11 22:10:15 dist-test-slave-vxj2 krb5kdc[8715](info): commencing operation
krb5kdc: starting...
W20260811 22:10:16.881642 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:10:16 dist-test-slave-vxj2 krb5kdc[8715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:10:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:10:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:10:17.028944 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:35289
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:36319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:35289
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:10:17.154557 8732 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:10:17.154964 8732 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:10:17.155049 8732 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:10:17.159379 8732 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:10:17.159520 8732 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:10:17.159559 8732 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:10:17.159602 8732 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:10:17.159636 8732 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:10:17.167522 8732 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:36319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:35289
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:35289
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8732
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:10:17.168900 8732 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:10:17.170192 8732 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:10:17.181090 8739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:17.181080 8741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:17.181269 8732 server_base.cc:1034] running on GCE node
W20260811 22:10:17.181087 8738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:17.182235 8732 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:10:17.183707 8732 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:10:17.184947 8732 hybrid_clock.cc:648] HybridClock initialized: now 1786486217184924 us; error 42 us; skew 500 ppm
Aug 11 22:10:17 dist-test-slave-vxj2 krb5kdc[8715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486217, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:10:17.188275 8732 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:10:17.189507 8732 webserver.cc:460] Webserver started at http://127.28.47.190:43045/ using document root <none> and password file <none>
I20260811 22:10:17.190155 8732 fs_manager.cc:359] Metadata directory not provided
I20260811 22:10:17.190230 8732 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:10:17.190435 8732 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:10:17.192768 8732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "3cac611ecec3477ea8288308445ac96b"
format_stamp: "Formatted at 2026-08-11 22:10:17 on dist-test-slave-vxj2"
server_key: "4c93f811746041612d838081f7e11660"
server_key_iv: "63d3cf133d6eb02bd1621caa929aa8b1"
server_key_version: "encryptionkey@0"
I20260811 22:10:17.193482 8732 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "3cac611ecec3477ea8288308445ac96b"
format_stamp: "Formatted at 2026-08-11 22:10:17 on dist-test-slave-vxj2"
server_key: "4c93f811746041612d838081f7e11660"
server_key_iv: "63d3cf133d6eb02bd1621caa929aa8b1"
server_key_version: "encryptionkey@0"
I20260811 22:10:17.197978 8732 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:10:17.200857 8750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:17.202456 8732 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:10:17.202648 8732 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "3cac611ecec3477ea8288308445ac96b"
format_stamp: "Formatted at 2026-08-11 22:10:17 on dist-test-slave-vxj2"
server_key: "4c93f811746041612d838081f7e11660"
server_key_iv: "63d3cf133d6eb02bd1621caa929aa8b1"
server_key_version: "encryptionkey@0"
I20260811 22:10:17.202800 8732 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:10:17.226517 8732 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:10:17.230685 8732 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:10:17.230952 8732 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:10:17.239073 8732 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:35289
I20260811 22:10:17.239159 8811 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:35289 every 8 connection(s)
I20260811 22:10:17.240177 8732 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:10:17.243378 8813 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:10:17.247989 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 8732
I20260811 22:10:17.248097 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:10:17.248420 28862 external_mini_cluster.cc:1428] Setting key 66b9d23b5e4a6b4b07a9aaabddcb3c4a
I20260811 22:10:17.250211 8813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b: Bootstrap starting.
I20260811 22:10:17.253245 8813 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b: Neither blocks nor log segments found. Creating new log.
I20260811 22:10:17.254189 8813 log.cc:824] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:10:17.256922 8813 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b: No bootstrap required, opened a new log
Aug 11 22:10:17 dist-test-slave-vxj2 krb5kdc[8715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486216, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:10:17.260869 8813 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cac611ecec3477ea8288308445ac96b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35289 } }
I20260811 22:10:17.261274 8813 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:10:17.261375 8813 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cac611ecec3477ea8288308445ac96b, State: Initialized, Role: FOLLOWER
I20260811 22:10:17.262002 8813 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cac611ecec3477ea8288308445ac96b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35289 } }
I20260811 22:10:17.262207 8813 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:10:17.262291 8813 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:10:17.262418 8813 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:10:17.263854 8813 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cac611ecec3477ea8288308445ac96b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35289 } }
I20260811 22:10:17.264318 8813 leader_election.cc:302] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cac611ecec3477ea8288308445ac96b; no voters:
I20260811 22:10:17.264786 8813 leader_election.cc:288] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:10:17.264987 8818 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:10:17.265316 8818 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [term 1 LEADER]: Becoming Leader. State: Replica: 3cac611ecec3477ea8288308445ac96b, State: Running, Role: LEADER
I20260811 22:10:17.265779 8818 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cac611ecec3477ea8288308445ac96b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35289 } }
I20260811 22:10:17.266405 8813 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:10:17.266974 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:17.250068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56014 (local address 127.28.47.190:35289)
0811 22:10:17.250557 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:17.250574 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:17.250611 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:10:17.251313 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:17.251332 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:17.251689 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:17.252018 (+ 329us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:17.253086 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:17.254418 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:17.255493 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:17.256066 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:17.256223 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:17.259365 (+ 3142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:10:17.259371 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:10:17.259388 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:10:17.259423 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:10:17.262466 (+ 3043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:10:17.263034 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:10:17.263041 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:10:17.263048 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:10:17.263214 (+ 166us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:10:17.263574 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:10:17.263578 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:10:17.263580 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:10:17.264239 (+ 659us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:10:17.264464 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:17.264980 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 22:10:17.265654 (+ 674us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":327,"thread_start_us":183,"threads_started":1}
I20260811 22:10:17.268740 8820 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cac611ecec3477ea8288308445ac96b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cac611ecec3477ea8288308445ac96b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35289 } } }
I20260811 22:10:17.268900 8820 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [sys.catalog]: This master's current role is: LEADER
I20260811 22:10:17.269261 8819 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cac611ecec3477ea8288308445ac96b. Latest consensus state: current_term: 1 leader_uuid: "3cac611ecec3477ea8288308445ac96b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cac611ecec3477ea8288308445ac96b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35289 } } }
I20260811 22:10:17.269371 8819 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b [sys.catalog]: This master's current role is: LEADER
W20260811 22:10:17.272259 8840 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:10:17.272395 8840 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:10:17.272482 8841 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:10:17.273341 8841 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:10:17.281086 8841 catalog_manager.cc:1347] Generated new cluster ID: 02c60cdc521b4194b15e42710ba63875
I20260811 22:10:17.281205 8841 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:10:17.313861 8841 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:10:17.315263 8841 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:10:17.321219 8841 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b: Generated new TSK 0
I20260811 22:10:17.322038 8841 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:10:17.361766 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35289
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:36319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:10:17.489310 8850 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:10:17.489657 8850 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:10:17.489713 8850 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:10:17.494246 8850 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:10:17.494341 8850 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:10:17.494406 8850 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:10:17.500697 8850 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:36319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8850
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:10:17.502148 8850 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:10:17.503412 8850 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:10:17.514745 8860 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:17.514873 8856 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:17.514760 8857 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:17.515038 8850 server_base.cc:1034] running on GCE node
I20260811 22:10:17.515609 8850 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:10:17.516551 8850 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:10:17.517792 8850 hybrid_clock.cc:648] HybridClock initialized: now 1786486217517774 us; error 28 us; skew 500 ppm
Aug 11 22:10:17 dist-test-slave-vxj2 krb5kdc[8715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486217, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:10:17.524941 8850 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:10:17.526172 8850 webserver.cc:460] Webserver started at http://127.28.47.129:37215/ using document root <none> and password file <none>
I20260811 22:10:17.526860 8850 fs_manager.cc:359] Metadata directory not provided
I20260811 22:10:17.526968 8850 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:10:17.527226 8850 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:10:17.529627 8850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "4d0c9aa15645433584043e2644f24ede"
format_stamp: "Formatted at 2026-08-11 22:10:17 on dist-test-slave-vxj2"
server_key: "978b436ccfd2511ace17bd32c5282406"
server_key_iv: "6ed4f431ceabc130b15007cf866cc1c6"
server_key_version: "encryptionkey@0"
I20260811 22:10:17.530274 8850 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "4d0c9aa15645433584043e2644f24ede"
format_stamp: "Formatted at 2026-08-11 22:10:17 on dist-test-slave-vxj2"
server_key: "978b436ccfd2511ace17bd32c5282406"
server_key_iv: "6ed4f431ceabc130b15007cf866cc1c6"
server_key_version: "encryptionkey@0"
I20260811 22:10:17.534487 8850 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:10:17.537403 8868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:17.538904 8850 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:10:17.539088 8850 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "4d0c9aa15645433584043e2644f24ede"
format_stamp: "Formatted at 2026-08-11 22:10:17 on dist-test-slave-vxj2"
server_key: "978b436ccfd2511ace17bd32c5282406"
server_key_iv: "6ed4f431ceabc130b15007cf866cc1c6"
server_key_version: "encryptionkey@0"
I20260811 22:10:17.539233 8850 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:10:17.565734 8850 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:10:17.569805 8850 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:10:17.570022 8850 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:10:17.570657 8850 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:10:17.571502 8850 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:10:17.571550 8850 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:17.571591 8850 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:10:17.571607 8850 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:17.581561 8850 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:37063
I20260811 22:10:17.581691 9004 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:37063 every 8 connection(s)
I20260811 22:10:17.582626 8850 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:10:17.590870 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 8850
I20260811 22:10:17.591048 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:10:17.591363 28862 external_mini_cluster.cc:1428] Setting key bda16946e5f87b30e43d9718ef020e2c
Aug 11 22:10:17 dist-test-slave-vxj2 krb5kdc[8715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486217, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:10:17.598912 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:17.584682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:51761 (local address 127.28.47.190:35289)
0811 22:10:17.584826 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:17.584831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:17.585947 (+ 1116us) server_negotiation.cc:407] Connection header received
0811 22:10:17.587845 (+ 1898us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:17.587849 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:17.587921 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:17.588039 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:17.589482 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:17.590040 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:17.591461 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:17.591743 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:17.591882 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:17.595003 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:10:17.595006 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:10:17.595009 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:10:17.595036 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:10:17.596582 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:10:17.597218 (+ 636us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:10:17.597229 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:10:17.597231 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:10:17.597314 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:10:17.597698 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:10:17.597701 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:10:17.597703 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:10:17.597918 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:10:17.598031 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:17.598593 (+ 562us) server_negotiation.cc:299] Negotiation successful
0811 22:10:17.598739 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:10:17.599895 9009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:17.585104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:35289 (local address 127.28.47.129:51761)
0811 22:10:17.585738 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:17.585789 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:17.587550 (+ 1761us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:17.588258 (+ 708us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:17.588266 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:17.588685 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:17.589312 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:17.589329 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:17.590196 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:17.590199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:17.591317 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:17.591329 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:17.592009 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:17.592017 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:17.592276 (+ 259us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:17.593012 (+ 736us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:17.593043 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:17.594783 (+ 1740us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:10:17.596738 (+ 1955us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:10:17.596744 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:10:17.596764 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:10:17.597076 (+ 312us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:10:17.597414 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:10:17.597418 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:10:17.597423 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:10:17.597590 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:10:17.598031 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:10:17.598039 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:10:17.598335 (+ 296us) client_negotiation.cc:769] Sending connection context
0811 22:10:17.598535 (+ 200us) client_negotiation.cc:240] Negotiation successful
0811 22:10:17.598928 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":426,"thread_start_us":144,"threads_started":1}
I20260811 22:10:17.601430 9007 heartbeater.cc:347] Connected to a master server at 127.28.47.190:35289
I20260811 22:10:17.601756 9007 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
I20260811 22:10:17.602501 9007 heartbeater.cc:511] Master 127.28.47.190:35289 requested a full tablet report, sending...
I20260811 22:10:17.604875 8769 ts_manager.cc:194] Registered new tserver with Master: 4d0c9aa15645433584043e2644f24ede (127.28.47.129:37063)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:10:17.606442 8769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:51761
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:10:17.625382 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35289
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:36319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:10:17.748834 9013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:10:17.749244 9013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:10:17.749305 9013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:10:17.753521 9013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:10:17.753593 9013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:10:17.753656 9013 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:10:17.759272 9013 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:36319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:10:17.760458 9013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:10:17.761554 9013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:10:17.773376 9020 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:17.773351 9018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:17.773347 9022 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:17.773811 9013 server_base.cc:1034] running on GCE node
I20260811 22:10:17.774284 9013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:10:17.774991 9013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:10:17.776203 9013 hybrid_clock.cc:648] HybridClock initialized: now 1786486217776182 us; error 37 us; skew 500 ppm
Aug 11 22:10:17 dist-test-slave-vxj2 krb5kdc[8715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486217, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:10:17.779516 9013 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:10:17.780685 9013 webserver.cc:460] Webserver started at http://127.28.47.130:43771/ using document root <none> and password file <none>
I20260811 22:10:17.781416 9013 fs_manager.cc:359] Metadata directory not provided
I20260811 22:10:17.781518 9013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:10:17.781765 9013 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:10:17.784073 9013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "de73b738c4ef4d329c12c7290fe0330e"
format_stamp: "Formatted at 2026-08-11 22:10:17 on dist-test-slave-vxj2"
server_key: "35c9bc71968aa0b23112e568bbd892d3"
server_key_iv: "5366aec5ced3e2ed110c313cafc6c49c"
server_key_version: "encryptionkey@0"
I20260811 22:10:17.784708 9013 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "de73b738c4ef4d329c12c7290fe0330e"
format_stamp: "Formatted at 2026-08-11 22:10:17 on dist-test-slave-vxj2"
server_key: "35c9bc71968aa0b23112e568bbd892d3"
server_key_iv: "5366aec5ced3e2ed110c313cafc6c49c"
server_key_version: "encryptionkey@0"
I20260811 22:10:17.789075 9013 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 22:10:17.792563 9029 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:17.794375 9013 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:10:17.794589 9013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "de73b738c4ef4d329c12c7290fe0330e"
format_stamp: "Formatted at 2026-08-11 22:10:17 on dist-test-slave-vxj2"
server_key: "35c9bc71968aa0b23112e568bbd892d3"
server_key_iv: "5366aec5ced3e2ed110c313cafc6c49c"
server_key_version: "encryptionkey@0"
I20260811 22:10:17.794764 9013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:10:17.816690 9013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:10:17.820763 9013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:10:17.821012 9013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:10:17.821760 9013 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:10:17.822714 9013 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:10:17.822809 9013 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:17.822882 9013 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:10:17.822930 9013 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:17.835435 9013 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:33863
I20260811 22:10:17.835465 9166 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:33863 every 8 connection(s)
I20260811 22:10:17.836454 9013 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:10:17.844295 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 9013
I20260811 22:10:17.844471 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:10:17.844794 28862 external_mini_cluster.cc:1428] Setting key 1fe3965bbca08a981b38cf4291f2b8f9
Aug 11 22:10:17 dist-test-slave-vxj2 krb5kdc[8715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486217, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:10:17.852335 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:17.838607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:52321 (local address 127.28.47.190:35289)
0811 22:10:17.838766 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:17.838772 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:17.839754 (+ 982us) server_negotiation.cc:407] Connection header received
0811 22:10:17.841222 (+ 1468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:17.841225 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:17.841278 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:17.841356 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:17.842688 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:17.843338 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:17.844616 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:17.844964 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:17.845081 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:17.848192 (+ 3111us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:10:17.848198 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:10:17.848205 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:10:17.848245 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:10:17.849993 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:10:17.850594 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:10:17.850599 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:10:17.850600 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:10:17.850702 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:10:17.851106 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:10:17.851109 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:10:17.851111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:10:17.851341 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:10:17.851454 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:17.852013 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 22:10:17.852162 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:10:17.853338 9169 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:17.839085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:35289 (local address 127.28.47.130:52321)
0811 22:10:17.839587 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:17.839633 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:17.840962 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:17.841529 (+ 567us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:17.841536 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:17.842006 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:17.842526 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:17.842541 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:17.843472 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:17.843475 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:17.844392 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:17.844406 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:17.845128 (+ 722us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:17.845131 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:17.845340 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:17.845983 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:17.846007 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:17.847976 (+ 1969us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:10:17.850121 (+ 2145us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:10:17.850126 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:10:17.850142 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:10:17.850465 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:10:17.850805 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:10:17.850808 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:10:17.850811 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:10:17.850992 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:10:17.851454 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:10:17.851463 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:10:17.851756 (+ 293us) client_negotiation.cc:769] Sending connection context
0811 22:10:17.851969 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 22:10:17.852352 (+ 383us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":133,"threads_started":1}
I20260811 22:10:17.854849 9167 heartbeater.cc:347] Connected to a master server at 127.28.47.190:35289
I20260811 22:10:17.855155 9167 heartbeater.cc:464] Registering TS with master...
I20260811 22:10:17.855786 9167 heartbeater.cc:511] Master 127.28.47.190:35289 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:10:17.857111 8769 ts_manager.cc:194] Registered new tserver with Master: de73b738c4ef4d329c12c7290fe0330e (127.28.47.130:33863)
I20260811 22:10:17.857853 8769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:52321
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:10:17.880878 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35289
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:36319
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:10:18.003413 9176 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:10:18.003731 9176 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:10:18.003804 9176 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:10:18.008349 9176 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:10:18.008430 9176 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:10:18.008494 9176 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:10:18.014205 9176 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:36319
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35289
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9176
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:10:18.015412 9176 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:10:18.016492 9176 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:10:18.027613 9183 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:18.027619 9185 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:10:18.027619 9182 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:10:18.027693 9176 server_base.cc:1034] running on GCE node
I20260811 22:10:18.028471 9176 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:10:18.029377 9176 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:10:18.030682 9176 hybrid_clock.cc:648] HybridClock initialized: now 1786486218030553 us; error 143 us; skew 500 ppm
Aug 11 22:10:18 dist-test-slave-vxj2 krb5kdc[8715](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486218, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:10:18.034648 9176 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:10:18.036190 9176 webserver.cc:460] Webserver started at http://127.28.47.131:45277/ using document root <none> and password file <none>
I20260811 22:10:18.036906 9176 fs_manager.cc:359] Metadata directory not provided
I20260811 22:10:18.036990 9176 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:10:18.037353 9176 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:10:18.039840 9176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "5717938feeaf482fb3355b576ad8b430"
format_stamp: "Formatted at 2026-08-11 22:10:18 on dist-test-slave-vxj2"
server_key: "be4ff341faf7af71ce78c2a112415d05"
server_key_iv: "d708121ff5930555fdb212ddb1f79aee"
server_key_version: "encryptionkey@0"
I20260811 22:10:18.040561 9176 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "5717938feeaf482fb3355b576ad8b430"
format_stamp: "Formatted at 2026-08-11 22:10:18 on dist-test-slave-vxj2"
server_key: "be4ff341faf7af71ce78c2a112415d05"
server_key_iv: "d708121ff5930555fdb212ddb1f79aee"
server_key_version: "encryptionkey@0"
I20260811 22:10:18.046164 9176 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.003s
I20260811 22:10:18.049515 9193 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:18.051192 9176 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:10:18.051349 9176 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "5717938feeaf482fb3355b576ad8b430"
format_stamp: "Formatted at 2026-08-11 22:10:18 on dist-test-slave-vxj2"
server_key: "be4ff341faf7af71ce78c2a112415d05"
server_key_iv: "d708121ff5930555fdb212ddb1f79aee"
server_key_version: "encryptionkey@0"
I20260811 22:10:18.051573 9176 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:10:18.073737 9176 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:10:18.077925 9176 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:10:18.078184 9176 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:10:18.078933 9176 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:10:18.079967 9176 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:10:18.080022 9176 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:18.080071 9176 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:10:18.080127 9176 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:10:18.090822 9176 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:35625
I20260811 22:10:18.090816 9324 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:35625 every 8 connection(s)
I20260811 22:10:18.091921 9176 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:10:18.099812 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 9176
I20260811 22:10:18.099987 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:10:18.100447 28862 external_mini_cluster.cc:1428] Setting key 9465d96bd0dd855be452e88b386b772f
Aug 11 22:10:18 dist-test-slave-vxj2 krb5kdc[8715](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486218, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:10:18.111483 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.094379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:54715 (local address 127.28.47.190:35289)
0811 22:10:18.094563 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:18.094567 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:18.095571 (+ 1004us) server_negotiation.cc:407] Connection header received
0811 22:10:18.097137 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:18.097140 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:18.097229 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:18.097343 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:18.098845 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.099887 (+ 1042us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.101983 (+ 2096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.102384 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.102511 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.106202 (+ 3691us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:10:18.106206 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:10:18.106208 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:10:18.106242 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:10:18.108117 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:10:18.109010 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:10:18.109014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:10:18.109015 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:10:18.109117 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:10:18.109716 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:10:18.109719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:10:18.109721 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:10:18.109961 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:10:18.110056 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:18.110904 (+ 848us) server_negotiation.cc:299] Negotiation successful
0811 22:10:18.111192 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260811 22:10:18.112111 9328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.094791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:35289 (local address 127.28.47.131:54715)
0811 22:10:18.095361 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:18.095415 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:18.096854 (+ 1439us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:18.097540 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:18.097549 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:18.097990 (+ 441us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:10:18.098574 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:18.098591 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.100150 (+ 1559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.100156 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:18.101797 (+ 1641us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:18.101811 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.102571 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.102575 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:18.102814 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.103775 (+ 961us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:10:18.103795 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:10:18.105918 (+ 2123us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:10:18.108323 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:10:18.108330 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:10:18.108349 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:10:18.108847 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:10:18.109333 (+ 486us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:10:18.109336 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:10:18.109337 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:10:18.109570 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:10:18.110079 (+ 509us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:10:18.110086 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:10:18.110426 (+ 340us) client_negotiation.cc:769] Sending connection context
0811 22:10:18.110616 (+ 190us) client_negotiation.cc:240] Negotiation successful
0811 22:10:18.110981 (+ 365us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":134,"threads_started":1}
I20260811 22:10:18.113796 9325 heartbeater.cc:347] Connected to a master server at 127.28.47.190:35289
I20260811 22:10:18.114214 9325 heartbeater.cc:464] Registering TS with master...
I20260811 22:10:18.114921 9325 heartbeater.cc:511] Master 127.28.47.190:35289 requested a full tablet report, sending...
I20260811 22:10:18.116353 8769 ts_manager.cc:194] Registered new tserver with Master: 5717938feeaf482fb3355b576ad8b430 (127.28.47.131:35625)
I20260811 22:10:18.117117 8769 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:54715
I20260811 22:10:18.129136 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:10:18.140092 8816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.131603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56028 (local address 127.28.47.190:35289)
0811 22:10:18.131804 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:18.131810 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:18.131990 (+ 180us) server_negotiation.cc:407] Connection header received
0811 22:10:18.132167 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:18.132170 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:18.132224 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:18.132347 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:10:18.133187 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.133728 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.134612 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.134846 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.134962 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.136515 (+ 1553us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:10:18.136523 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:10:18.136525 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:10:18.136554 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:10:18.138156 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:10:18.138683 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:10:18.138687 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:10:18.138688 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:10:18.138779 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:10:18.139078 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:10:18.139082 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:10:18.139083 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:10:18.139343 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:10:18.139502 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:18.139754 (+ 252us) server_negotiation.cc:299] Negotiation successful
0811 22:10:18.139905 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 22:10:18.143793 8769 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56028:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:10:18.147275 8769 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:10:18.163988 9338 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.157713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:37063 (local address 127.0.0.1:34520)
0811 22:10:18.158166 (+ 453us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:18.158234 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:18.158420 (+ 186us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:18.159422 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:18.159429 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:18.159448 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:10:18.159606 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:18.159612 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.161062 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.161068 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:18.162148 (+ 1080us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:18.162158 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.163329 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.163333 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:18.163469 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.163529 (+ 60us) client_negotiation.cc:769] Sending connection context
0811 22:10:18.163654 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:10:18.163802 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":248,"mutex_wait_us":1,"thread_start_us":123,"threads_started":1}
I20260811 22:10:18.164000 9339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.157869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34520 (local address 127.28.47.129:37063)
0811 22:10:18.158162 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:18.158170 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:18.158335 (+ 165us) server_negotiation.cc:407] Connection header received
0811 22:10:18.158741 (+ 406us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:18.158748 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:18.159000 (+ 252us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:18.159194 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:10:18.159736 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.160945 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.162298 (+ 1353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.163189 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.163308 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.163512 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:18.163715 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 22:10:18.163830 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":187,"thread_start_us":100,"threads_started":1}
I20260811 22:10:18.165033 9340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.158553 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:35625 (local address 127.0.0.1:35384)
0811 22:10:18.159012 (+ 459us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:18.159029 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:18.159122 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:18.160380 (+ 1258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:18.160383 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:18.160411 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:10:18.160519 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:18.160529 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.162105 (+ 1576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.162109 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:18.163358 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:18.163370 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.164572 (+ 1202us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.164577 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:18.164666 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.164690 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:10:18.164770 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:10:18.164848 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":364,"mutex_wait_us":68,"thread_start_us":154,"threads_started":1}
I20260811 22:10:18.165061 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.158803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35384 (local address 127.28.47.131:35625)
0811 22:10:18.159990 (+ 1187us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:18.160002 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:18.160031 (+ 29us) server_negotiation.cc:407] Connection header received
0811 22:10:18.160092 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:18.160096 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:18.160237 (+ 141us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:18.160401 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:10:18.160657 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.161905 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.163513 (+ 1608us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.164419 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.164553 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.164709 (+ 156us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:18.164795 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 22:10:18.164904 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1067,"thread_start_us":226,"threads_started":1}
I20260811 22:10:18.165436 9342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.159088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:33863 (local address 127.0.0.1:41996)
0811 22:10:18.159344 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:18.159359 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:18.159442 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:18.160567 (+ 1125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:18.160570 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:18.160588 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:10:18.160666 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:18.160671 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.162331 (+ 1660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.162335 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:18.163404 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:18.163413 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.164802 (+ 1389us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.164805 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:18.164880 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.164899 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:10:18.164966 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:10:18.165030 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":105,"threads_started":1}
I20260811 22:10:18.165444 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.159476 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41996 (local address 127.28.47.130:33863)
0811 22:10:18.159916 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:18.159928 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:18.159952 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:10:18.160031 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:18.160037 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:18.160240 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:18.160402 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:10:18.160793 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.161942 (+ 1149us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.163558 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.164448 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.164588 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.164740 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:18.165109 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 22:10:18.165244 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":156,"threads_started":1}
I20260811 22:10:18.168053 9248 tablet_service.cc:1467] Processing CreateTablet for tablet ccbb53b9f87440039485f6e3d4dc54f1 (DEFAULT_TABLE table=test-table [id=52ae6c848f3441d59aa54700383d4a35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:10:18.168053 8923 tablet_service.cc:1467] Processing CreateTablet for tablet ccbb53b9f87440039485f6e3d4dc54f1 (DEFAULT_TABLE table=test-table [id=52ae6c848f3441d59aa54700383d4a35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:10:18.168284 9085 tablet_service.cc:1467] Processing CreateTablet for tablet ccbb53b9f87440039485f6e3d4dc54f1 (DEFAULT_TABLE table=test-table [id=52ae6c848f3441d59aa54700383d4a35]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:10:18.169384 9085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccbb53b9f87440039485f6e3d4dc54f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:10:18.169389 8923 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccbb53b9f87440039485f6e3d4dc54f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:10:18.169399 9248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ccbb53b9f87440039485f6e3d4dc54f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:10:18.177237 9346 tablet_bootstrap.cc:492] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede: Bootstrap starting.
I20260811 22:10:18.178156 9345 tablet_bootstrap.cc:492] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e: Bootstrap starting.
I20260811 22:10:18.178363 9347 tablet_bootstrap.cc:492] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430: Bootstrap starting.
I20260811 22:10:18.180047 9346 tablet_bootstrap.cc:654] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede: Neither blocks nor log segments found. Creating new log.
I20260811 22:10:18.180576 9345 tablet_bootstrap.cc:654] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e: Neither blocks nor log segments found. Creating new log.
I20260811 22:10:18.181005 9347 tablet_bootstrap.cc:654] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430: Neither blocks nor log segments found. Creating new log.
I20260811 22:10:18.181128 9346 log.cc:824] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede: Log is configured to *not* fsync() on all Append() calls
I20260811 22:10:18.181470 9345 log.cc:824] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e: Log is configured to *not* fsync() on all Append() calls
I20260811 22:10:18.181828 9347 log.cc:824] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430: Log is configured to *not* fsync() on all Append() calls
I20260811 22:10:18.183683 9346 tablet_bootstrap.cc:492] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede: No bootstrap required, opened a new log
I20260811 22:10:18.183741 9345 tablet_bootstrap.cc:492] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e: No bootstrap required, opened a new log
I20260811 22:10:18.183743 9347 tablet_bootstrap.cc:492] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430: No bootstrap required, opened a new log
I20260811 22:10:18.183954 9346 ts_tablet_manager.cc:1397] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260811 22:10:18.183952 9345 ts_tablet_manager.cc:1397] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:10:18.183952 9347 ts_tablet_manager.cc:1397] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:10:18.187703 9346 raft_consensus.cc:348] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0c9aa15645433584043e2644f24ede" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37063 } } peers { permanent_uuid: "5717938feeaf482fb3355b576ad8b430" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35625 } } peers { permanent_uuid: "de73b738c4ef4d329c12c7290fe0330e" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33863 } }
I20260811 22:10:18.188035 9346 raft_consensus.cc:374] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:10:18.188095 9346 raft_consensus.cc:729] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4d0c9aa15645433584043e2644f24ede, State: Initialized, Role: FOLLOWER
I20260811 22:10:18.188145 9347 raft_consensus.cc:348] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0c9aa15645433584043e2644f24ede" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37063 } } peers { permanent_uuid: "5717938feeaf482fb3355b576ad8b430" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35625 } } peers { permanent_uuid: "de73b738c4ef4d329c12c7290fe0330e" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33863 } }
I20260811 22:10:18.188419 9347 raft_consensus.cc:374] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:10:18.188488 9347 raft_consensus.cc:729] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5717938feeaf482fb3355b576ad8b430, State: Initialized, Role: FOLLOWER
I20260811 22:10:18.188146 9345 raft_consensus.cc:348] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0c9aa15645433584043e2644f24ede" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37063 } } peers { permanent_uuid: "5717938feeaf482fb3355b576ad8b430" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35625 } } peers { permanent_uuid: "de73b738c4ef4d329c12c7290fe0330e" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33863 } }
I20260811 22:10:18.188740 9345 raft_consensus.cc:374] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:10:18.188796 9345 raft_consensus.cc:729] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: de73b738c4ef4d329c12c7290fe0330e, State: Initialized, Role: FOLLOWER
I20260811 22:10:18.188674 9346 consensus_queue.cc:260] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0c9aa15645433584043e2644f24ede" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37063 } } peers { permanent_uuid: "5717938feeaf482fb3355b576ad8b430" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35625 } } peers { permanent_uuid: "de73b738c4ef4d329c12c7290fe0330e" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33863 } }
I20260811 22:10:18.189196 9347 consensus_queue.cc:260] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0c9aa15645433584043e2644f24ede" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37063 } } peers { permanent_uuid: "5717938feeaf482fb3355b576ad8b430" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35625 } } peers { permanent_uuid: "de73b738c4ef4d329c12c7290fe0330e" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33863 } }
I20260811 22:10:18.189343 9345 consensus_queue.cc:260] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0c9aa15645433584043e2644f24ede" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37063 } } peers { permanent_uuid: "5717938feeaf482fb3355b576ad8b430" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35625 } } peers { permanent_uuid: "de73b738c4ef4d329c12c7290fe0330e" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33863 } }
I20260811 22:10:18.190158 9346 ts_tablet_manager.cc:1428] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede: Time spent starting tablet: real 0.006s user 0.001s sys 0.007s
I20260811 22:10:18.190312 9345 ts_tablet_manager.cc:1428] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:10:18.190322 9347 ts_tablet_manager.cc:1428] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:10:18.190433 9007 heartbeater.cc:503] Master 127.28.47.190:35289 was elected leader, sending a full tablet report...
I20260811 22:10:18.190670 9167 heartbeater.cc:503] Master 127.28.47.190:35289 was elected leader, sending a full tablet report...
I20260811 22:10:18.190686 9325 heartbeater.cc:503] Master 127.28.47.190:35289 was elected leader, sending a full tablet report...
I20260811 22:10:18.207914 9354 raft_consensus.cc:482] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:10:18.208200 9354 raft_consensus.cc:504] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0c9aa15645433584043e2644f24ede" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37063 } } peers { permanent_uuid: "5717938feeaf482fb3355b576ad8b430" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35625 } } peers { permanent_uuid: "de73b738c4ef4d329c12c7290fe0330e" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33863 } }
I20260811 22:10:18.209851 9354 leader_election.cc:288] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 5717938feeaf482fb3355b576ad8b430 (127.28.47.131:35625), de73b738c4ef4d329c12c7290fe0330e (127.28.47.130:33863)
I20260811 22:10:18.214042 9341 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.210323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:49961 (local address 127.28.47.131:35625)
0811 22:10:18.210480 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:18.210484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:18.210609 (+ 125us) server_negotiation.cc:407] Connection header received
0811 22:10:18.210800 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:18.210806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:18.210868 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:18.210944 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:10:18.211362 (+ 418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.211875 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.212989 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.213461 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.213593 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.213650 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:18.213832 (+ 182us) server_negotiation.cc:299] Negotiation successful
0811 22:10:18.213904 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:10:18.214059 9357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.210170 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:35625 (local address 127.28.47.129:49961)
0811 22:10:18.210497 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:18.210515 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:18.210634 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:18.211071 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:18.211073 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:18.211092 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:10:18.211205 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:18.211210 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.212001 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.212007 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:18.212843 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:18.212853 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.213590 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.213594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:18.213699 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.213722 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:10:18.213815 (+ 93us) client_negotiation.cc:240] Negotiation successful
0811 22:10:18.213896 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":148,"threads_started":1}
I20260811 22:10:18.214627 9343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.210791 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:52817 (local address 127.28.47.130:33863)
0811 22:10:18.210956 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:18.210961 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:18.211159 (+ 198us) server_negotiation.cc:407] Connection header received
0811 22:10:18.211303 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:18.211308 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:18.211362 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:18.211469 (+ 107us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:10:18.211764 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.212313 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.213525 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.214021 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.214168 (+ 147us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.214222 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:18.214430 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 22:10:18.214509 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260811 22:10:18.214645 9358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.210606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:33863 (local address 127.28.47.129:52817)
0811 22:10:18.211043 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:18.211067 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0811 22:10:18.211203 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:10:18.211479 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:10:18.211482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:10:18.211499 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:10:18.211637 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:18.211644 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.212442 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.212448 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:18.213390 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:10:18.213400 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.214159 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.214162 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:10:18.214271 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.214306 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:10:18.214410 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 22:10:18.214486 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":187,"threads_started":1}
I20260811 22:10:18.215265 9272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccbb53b9f87440039485f6e3d4dc54f1" candidate_uuid: "4d0c9aa15645433584043e2644f24ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5717938feeaf482fb3355b576ad8b430" is_pre_election: true
I20260811 22:10:18.215673 9272 raft_consensus.cc:2454] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d0c9aa15645433584043e2644f24ede in term 0.
I20260811 22:10:18.215701 9106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccbb53b9f87440039485f6e3d4dc54f1" candidate_uuid: "4d0c9aa15645433584043e2644f24ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de73b738c4ef4d329c12c7290fe0330e" is_pre_election: true
I20260811 22:10:18.216074 9106 raft_consensus.cc:2454] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4d0c9aa15645433584043e2644f24ede in term 0.
I20260811 22:10:18.216352 8870 leader_election.cc:302] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d0c9aa15645433584043e2644f24ede, 5717938feeaf482fb3355b576ad8b430; no voters:
I20260811 22:10:18.216712 9354 raft_consensus.cc:2781] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:10:18.216853 9354 raft_consensus.cc:482] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:10:18.216992 9354 raft_consensus.cc:3037] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:10:18.218335 9354 raft_consensus.cc:504] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0c9aa15645433584043e2644f24ede" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37063 } } peers { permanent_uuid: "5717938feeaf482fb3355b576ad8b430" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35625 } } peers { permanent_uuid: "de73b738c4ef4d329c12c7290fe0330e" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33863 } }
I20260811 22:10:18.218902 9354 leader_election.cc:288] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [CANDIDATE]: Term 1 election: Requested vote from peers 5717938feeaf482fb3355b576ad8b430 (127.28.47.131:35625), de73b738c4ef4d329c12c7290fe0330e (127.28.47.130:33863)
I20260811 22:10:18.219429 9272 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccbb53b9f87440039485f6e3d4dc54f1" candidate_uuid: "4d0c9aa15645433584043e2644f24ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5717938feeaf482fb3355b576ad8b430"
I20260811 22:10:18.219506 9106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ccbb53b9f87440039485f6e3d4dc54f1" candidate_uuid: "4d0c9aa15645433584043e2644f24ede" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "de73b738c4ef4d329c12c7290fe0330e"
I20260811 22:10:18.219573 9272 raft_consensus.cc:3037] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:10:18.219714 9106 raft_consensus.cc:3037] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:10:18.221138 9106 raft_consensus.cc:2454] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d0c9aa15645433584043e2644f24ede in term 1.
I20260811 22:10:18.221138 9272 raft_consensus.cc:2454] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4d0c9aa15645433584043e2644f24ede in term 1.
I20260811 22:10:18.221637 8870 leader_election.cc:302] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4d0c9aa15645433584043e2644f24ede, de73b738c4ef4d329c12c7290fe0330e; no voters:
I20260811 22:10:18.221935 9354 raft_consensus.cc:2781] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:10:18.222250 9354 raft_consensus.cc:686] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [term 1 LEADER]: Becoming Leader. State: Replica: 4d0c9aa15645433584043e2644f24ede, State: Running, Role: LEADER
I20260811 22:10:18.222674 9354 consensus_queue.cc:237] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0c9aa15645433584043e2644f24ede" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37063 } } peers { permanent_uuid: "5717938feeaf482fb3355b576ad8b430" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35625 } } peers { permanent_uuid: "de73b738c4ef4d329c12c7290fe0330e" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33863 } }
I20260811 22:10:18.227191 8768 catalog_manager.cc:5495] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede reported cstate change: term changed from 0 to 1, leader changed from <none> to 4d0c9aa15645433584043e2644f24ede (127.28.47.129). New cstate: current_term: 1 leader_uuid: "4d0c9aa15645433584043e2644f24ede" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "4d0c9aa15645433584043e2644f24ede" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 37063 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "5717938feeaf482fb3355b576ad8b430" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35625 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "de73b738c4ef4d329c12c7290fe0330e" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33863 } health_report { overall_health: UNKNOWN } } }
I20260811 22:10:18.249570 9339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.245012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34524 (local address 127.28.47.129:37063)
0811 22:10:18.245135 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0811 22:10:18.245139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:10:18.245197 (+ 58us) server_negotiation.cc:407] Connection header received
0811 22:10:18.245310 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:10:18.245318 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:10:18.245379 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:10:18.245471 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:10:18.245738 (+ 267us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.246238 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.247719 (+ 1481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:10:18.248012 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:10:18.248116 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:10:18.248519 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:10:18.249063 (+ 544us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:10:18.249273 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0811 22:10:18.249306 (+ 33us) server_negotiation.cc:299] Negotiation successful
0811 22:10:18.249373 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 22:10:18.257438 9272 raft_consensus.cc:1261] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430 [term 1 FOLLOWER]: Refusing update from remote peer 4d0c9aa15645433584043e2644f24ede: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:10:18.257467 9106 raft_consensus.cc:1261] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e [term 1 FOLLOWER]: Refusing update from remote peer 4d0c9aa15645433584043e2644f24ede: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:10:18.258343 9359 consensus_queue.cc:1035] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [LEADER]: Connected to new peer: Peer: permanent_uuid: "5717938feeaf482fb3355b576ad8b430" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35625 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:10:18.258715 9354 consensus_queue.cc:1035] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [LEADER]: Connected to new peer: Peer: permanent_uuid: "de73b738c4ef4d329c12c7290fe0330e" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 33863 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:10:18.268252 9365 mvcc.cc:205] Tried to move back new op lower bound from 7317447549972557824 to 7317447549845540864. Current Snapshot: MvccSnapshot[applied={T|T < 7317447549972557824}]
I20260811 22:10:18.269182 9364 mvcc.cc:205] Tried to move back new op lower bound from 7317447549972557824 to 7317447549845540864. Current Snapshot: MvccSnapshot[applied={T|T < 7317447549972557824}]
I20260811 22:10:18.279134 8768 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:10:18.279385 8768 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:56028:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:10:18.283310 8768 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3cac611ecec3477ea8288308445ac96b: Sending DeleteTablet for 3 replicas of tablet ccbb53b9f87440039485f6e3d4dc54f1
I20260811 22:10:18.284705 8923 tablet_service.cc:1514] Processing DeleteTablet for tablet ccbb53b9f87440039485f6e3d4dc54f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:10:18 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:34520
I20260811 22:10:18.284693 9085 tablet_service.cc:1514] Processing DeleteTablet for tablet ccbb53b9f87440039485f6e3d4dc54f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:10:18 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:41996
I20260811 22:10:18.284700 9248 tablet_service.cc:1514] Processing DeleteTablet for tablet ccbb53b9f87440039485f6e3d4dc54f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:10:18 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:35384
I20260811 22:10:18.285413 9375 tablet_replica.cc:330] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede: stopping tablet replica
I20260811 22:10:18.285584 9374 tablet_replica.cc:330] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430: stopping tablet replica
I20260811 22:10:18.285840 9375 raft_consensus.cc:2229] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:10:18.285972 9374 raft_consensus.cc:2229] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:10:18.285902 9373 tablet_replica.cc:330] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e: stopping tablet replica
I20260811 22:10:18.286227 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 8850
I20260811 22:10:18.286362 9375 raft_consensus.cc:2258] T ccbb53b9f87440039485f6e3d4dc54f1 P 4d0c9aa15645433584043e2644f24ede [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:10:18.286392 9374 raft_consensus.cc:2258] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:10:18.297936 9373 raft_consensus.cc:2229] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:10:18.298529 9373 raft_consensus.cc:2258] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:10:18.300144 9374 ts_tablet_manager.cc:1905] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:10:18.301295 9373 ts_tablet_manager.cc:1905] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:10:18.303709 9374 ts_tablet_manager.cc:1918] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:10:18.303946 9374 log.cc:1196] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/ccbb53b9f87440039485f6e3d4dc54f1
I20260811 22:10:18.304522 9374 ts_tablet_manager.cc:1939] T ccbb53b9f87440039485f6e3d4dc54f1 P 5717938feeaf482fb3355b576ad8b430: Deleting consensus metadata
I20260811 22:10:18.304857 9373 ts_tablet_manager.cc:1918] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:10:18.304977 9373 log.cc:1196] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/ccbb53b9f87440039485f6e3d4dc54f1
I20260811 22:10:18.305349 9373 ts_tablet_manager.cc:1939] T ccbb53b9f87440039485f6e3d4dc54f1 P de73b738c4ef4d329c12c7290fe0330e: Deleting consensus metadata
I20260811 22:10:18.306277 8752 catalog_manager.cc:4841] TS 5717938feeaf482fb3355b576ad8b430 (127.28.47.131:35625): tablet ccbb53b9f87440039485f6e3d4dc54f1 (table test-table [id=52ae6c848f3441d59aa54700383d4a35]) successfully deleted
I20260811 22:10:18.307154 8752 catalog_manager.cc:4841] TS de73b738c4ef4d329c12c7290fe0330e (127.28.47.130:33863): tablet ccbb53b9f87440039485f6e3d4dc54f1 (table test-table [id=52ae6c848f3441d59aa54700383d4a35]) successfully deleted
W20260811 22:10:18.308763 8752 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:37063 (error 108)
I20260811 22:10:18.309826 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 9013
I20260811 22:10:18.310154 9342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:10:18.309750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:37063 (local address 127.0.0.1:34534)
0811 22:10:18.309921 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 22:10:18.310034 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:37063: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260811 22:10:18.312227 8752 catalog_manager.cc:4584] TS 4d0c9aa15645433584043e2644f24ede (127.28.47.129:37063): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ccbb53b9f87440039485f6e3d4dc54f1: Network error: Client connection negotiation failed: client connection to 127.28.47.129:37063: connect: Connection refused (error 111)
I20260811 22:10:18.318331 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 9176
I20260811 22:10:18.328089 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 8732
2026-08-11T22:10:18Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2594 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:10:18 dist-test-slave-vxj2 krb5kdc[9379](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:10:18 dist-test-slave-vxj2 krb5kdc[9379](info): set up 2 sockets
Aug 11 22:10:18 dist-test-slave-vxj2 krb5kdc[9379](info): commencing operation
krb5kdc: starting...
W20260811 22:10:19.508615 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:10:19 dist-test-slave-vxj2 krb5kdc[9379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486219, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:10:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:10:19Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.28.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.28.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/rangeradmin_127.28.47.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/rangeradmin_127.28.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.28.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.28.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/rangerlookup_127.28.47.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/rangerlookup_127.28.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.28.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/HTTP_127.28.47.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/HTTP_127.28.47.148@KRBTEST.COM.keytab.
I20260811 22:10:19.713505 28862 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktpwotM/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/postgres -l logfile start
2026-08-11 22:10:22.897 UTC [9414] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:10:22.898 UTC [9414] LOG: listening on IPv4 address "127.28.47.148", port 53645
2026-08-11 22:10:22.903 UTC [9414] LOG: listening on Unix socket "/tmp/.s.PGSQL.53645"
2026-08-11 22:10:22.930 UTC [9418] LOG: database system was shut down at 2026-08-11 22:10:20 UTC
2026-08-11 22:10:22.935 UTC [9414] LOG: database system is ready to accept connections
I20260811 22:10:24.014317 28862 mini_postgres.cc:96] Postgres bound to 53645
2026-08-11 22:10:24.026 UTC [9426] FATAL: database "slave" does not exist
127.28.47.148:53645 - accepting connections
I20260811 22:10:24.027001 28862 mini_ranger.cc:161] Starting Ranger...
I20260811 22:10:24.045542 28862 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:10:24.687945 28862 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:10:24.688072 28862 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-admin
2026-08-11 22:10:25,044 [I] DB FLAVOR :POSTGRES
2026-08-11 22:10:25,045 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:10:25,045 [I] Checking connection..
2026-08-11 22:10:25,045 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:10:25,342 [I] Checking connection passed.
2026-08-11 22:10:25,342 [I] --------- Verifying version history table ---------
2026-08-11 22:10:25,342 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:10:25,619 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:10:25,620 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:10:25,913 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:10:25,913 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:10:25,914 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:10:26.173 UTC [9504] WARNING: there is no transaction in progress
2026-08-11 22:10:26,182 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:10:26,182 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:10:26,472 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:10:26,472 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:10:26,473 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:10:26,755 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:10:27,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vxj2.c.gcp-upstream.internal','N') ;"
2026-08-11 22:10:27,274 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:10:27,560 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:10:27,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:10:27,853 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:10:27,854 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:10:28,137 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:10:28,420 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:10:28,420 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:10:29.625 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.647 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.670 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.689 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.710 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.732 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.801 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.810 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.830 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.843 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.855 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.873 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.886 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.895 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.904 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:29.914 UTC [9631] WARNING: there is no transaction in progress
2026-08-11 22:10:30,858 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:10:30,858 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:10:31,138 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:10:31,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:10:31,410 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:10:31,410 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:10:31,679 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:10:31,680 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:10:31,930 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:10:32,196 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vxj2.c.gcp-upstream.internal';"
2026-08-11 22:10:32,453 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:10:32,454 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:10:32,745 [I] DB_PATCHES have already been applied
I20260811 22:10:32.751029 28862 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktpwotM/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:10:32.751119 28862 mini_ranger.cc:193] Using host: 127.28.47.148
I20260811 22:10:32.755931 28862 mini_ranger.cc:227] Ranger admin URL: http://127.28.47.148:34195
Aug 11, 2026 10:10:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:10:33 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:10:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:10:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:10:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:10:33 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:10:33 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 22:10:34.049211 28862 mini_ranger.cc:160] Time spent starting Ranger: real 10.022s user 0.002s sys 0.012s
Aug 11, 2026 10:10:58 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:10:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.MetricsREST
Aug 11, 2026 10:10:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:10:58 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:10:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:10:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:10:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:10:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:10:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:10:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:10:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:10:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:10:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:10:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:10:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:10:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:10:58 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:10:59 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:11:00.387981 28862 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.28.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.28.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/rangerkms_127.28.47.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/rangerkms_127.28.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.47.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.28.47.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.47.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/HTTP_127.28.47.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.47.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/HTTP_127.28.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 22:11:00 dist-test-slave-vxj2 krb5kdc[9379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486260, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 22:11:00.508088 28862 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 22:11:00.525992 28862 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 22:11:01.681387 28862 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 22:11:01.681478 28862 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-kms
I20260811 22:11:01.681596 28862 mini_ranger_kms.cc:227] Using postgres at 127.28.47.148:53645
2026-08-11 22:11:02,886 [I] DB FLAVOR :POSTGRES
2026-08-11 22:11:02,886 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:11:02,887 [I] Checking connection
2026-08-11 22:11:02,887 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 22:11:03,271 [I] connection success
2026-08-11 22:11:03,272 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 22:11:03,272 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 22:11:03,272 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 22:11:03,555 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 22:11:03,555 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:11:03,556 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 22:11:03,556 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:53645/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 22:11:03,867 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 22:11:04.400427 28862 mini_ranger_kms.cc:310] Created kms service
I20260811 22:11:04.627153 28862 mini_ranger_kms.cc:326] Created kudu user
I20260811 22:11:04.674266 28862 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 22:11:04 dist-test-slave-vxj2 krb5kdc[9379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486260, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:11:05.153409 28862 mini_ranger_kms.cc:384] Added ranger policy
I20260811 22:11:05.153596 28862 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 22:11:05.153630 28862 mini_ranger_kms.cc:240] Using host: 127.28.47.148
I20260811 22:11:05.158631 28862 mini_ranger_kms.cc:276] Ranger KMS PID: 9850
I20260811 22:11:05.158811 28862 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.28.47.148:42023
Aug 11, 2026 10:11:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:11:06 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:11:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:11:06 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:11:07 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:11:07 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260811 22:11:07.720499 28862 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 7.212s user 0.018s sys 0.853s
I20260811 22:11:07.720656 28862 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 22:11:07.720774 28862 mini_ranger_kms.cc:401] 127.28.47.148:42023/kms/v1/keys
Aug 11 22:11:08 dist-test-slave-vxj2 krb5kdc[9379](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486268, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.47.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Aug 11 22:11:08 dist-test-slave-vxj2 krb5kdc[9379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486268, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.47.148@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
Aug 11 22:11:09 dist-test-slave-vxj2 krb5kdc[9379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486268, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.47.148@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
Aug 11 22:11:12 dist-test-slave-vxj2 krb5kdc[9379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:11:12.192430 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:38839
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:38839
--ranger_config_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.47.148:42023/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:12.379519 9914 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:12.379884 9914 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:12.379930 9914 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:12.384692 9914 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:11:12.384796 9914 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:12.384817 9914 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:12.384841 9914 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:11:12.384857 9914 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:11:12.391522 9914 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/wal
--ranger_kms_url=127.28.47.148:42023/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:38839
--ranger_config_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:38839
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9914
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:11:12.393441 9914 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:12.394774 9914 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:12.407222 9920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:12.407266 9923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:12.407462 9914 server_base.cc:1034] running on GCE node
W20260811 22:11:12.407203 9921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:12.408643 9914 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:12.410524 9914 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:12.411849 9914 hybrid_clock.cc:648] HybridClock initialized: now 1786486272411810 us; error 68 us; skew 500 ppm
Aug 11 22:11:12 dist-test-slave-vxj2 krb5kdc[9379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486272, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:12.420697 9914 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:11:12.422065 9914 webserver.cc:460] Webserver started at http://127.28.47.190:46251/ using document root <none> and password file <none>
I20260811 22:11:12.422770 9914 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:12.422866 9914 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:12.423110 9914 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:11:12 dist-test-slave-vxj2 krb5kdc[9379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486272, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:11:12.499076 9914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "473b2cd51b3342bf8de0165a9a3cf126"
format_stamp: "Formatted at 2026-08-11 22:11:12 on dist-test-slave-vxj2"
server_key: "570cf054e4bc95144cba986ddcb7efd6"
server_key_iv: "0047e11ab97881e0993801e639efd5df"
server_key_version: "kuduclusterkey@0"
I20260811 22:11:12.499861 9914 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "473b2cd51b3342bf8de0165a9a3cf126"
format_stamp: "Formatted at 2026-08-11 22:11:12 on dist-test-slave-vxj2"
server_key: "570cf054e4bc95144cba986ddcb7efd6"
server_key_iv: "0047e11ab97881e0993801e639efd5df"
server_key_version: "kuduclusterkey@0"
I20260811 22:11:12.504805 9914 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.001s sys 0.005s
I20260811 22:11:12.528123 9931 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:12.530045 9914 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.004s sys 0.000s
I20260811 22:11:12.530234 9914 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "473b2cd51b3342bf8de0165a9a3cf126"
format_stamp: "Formatted at 2026-08-11 22:11:12 on dist-test-slave-vxj2"
server_key: "570cf054e4bc95144cba986ddcb7efd6"
server_key_iv: "0047e11ab97881e0993801e639efd5df"
server_key_version: "kuduclusterkey@0"
I20260811 22:11:12.530386 9914 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:12.556602 9914 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:12.560348 9914 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:12.560905 9914 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:12.575659 9914 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:38839
I20260811 22:11:12.575765 9990 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:38839 every 8 connection(s)
I20260811 22:11:12.576967 9914 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:11:12.578944 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 9914
I20260811 22:11:12.579121 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:11:12.579450 28862 external_mini_cluster.cc:1428] Setting key 7d26da7ece96bf3e6690b247f69dc5fc
I20260811 22:11:12.581424 9991 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 11 22:11:12 dist-test-slave-vxj2 krb5kdc[9379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486272, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:11:12.591935 9991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126: Bootstrap starting.
I20260811 22:11:12.596120 9991 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:12.597887 9995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:12.581267 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44402 (local address 127.28.47.190:38839)
0811 22:11:12.582093 (+ 826us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:12.582151 (+ 58us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:12.582196 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:11:12.583381 (+ 1185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:12.583405 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:12.583802 (+ 397us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:12.584203 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:12.585218 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:12.586844 (+ 1626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:12.588033 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:12.588686 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:12.588895 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:12.592234 (+ 3339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:12.592240 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:12.592257 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:12.592296 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:12.594456 (+ 2160us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:12.594991 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:12.595003 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:12.595009 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:12.595201 (+ 192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:12.595512 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:12.595515 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:12.595517 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:12.596012 (+ 495us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:12.596381 (+ 369us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:12.596575 (+ 194us) server_negotiation.cc:299] Negotiation successful
0811 22:11:12.596920 (+ 345us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":643,"thread_start_us":364,"threads_started":1}
I20260811 22:11:12.598455 9991 log.cc:824] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:12.600987 9991 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126: No bootstrap required, opened a new log
I20260811 22:11:12.606066 9991 raft_consensus.cc:348] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473b2cd51b3342bf8de0165a9a3cf126" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38839 } }
I20260811 22:11:12.606410 9991 raft_consensus.cc:374] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:12.606510 9991 raft_consensus.cc:729] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 473b2cd51b3342bf8de0165a9a3cf126, State: Initialized, Role: FOLLOWER
I20260811 22:11:12.607115 9991 consensus_queue.cc:260] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473b2cd51b3342bf8de0165a9a3cf126" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38839 } }
I20260811 22:11:12.607380 9991 raft_consensus.cc:388] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:11:12.607465 9991 raft_consensus.cc:482] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:11:12.607585 9991 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:12.608937 9991 raft_consensus.cc:504] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473b2cd51b3342bf8de0165a9a3cf126" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38839 } }
I20260811 22:11:12.609395 9991 leader_election.cc:302] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 473b2cd51b3342bf8de0165a9a3cf126; no voters:
I20260811 22:11:12.609812 9991 leader_election.cc:288] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:11:12.610018 9997 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:12.610466 9997 raft_consensus.cc:686] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [term 1 LEADER]: Becoming Leader. State: Replica: 473b2cd51b3342bf8de0165a9a3cf126, State: Running, Role: LEADER
I20260811 22:11:12.611053 9997 consensus_queue.cc:237] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473b2cd51b3342bf8de0165a9a3cf126" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38839 } }
I20260811 22:11:12.611824 9991 sys_catalog.cc:564] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:11:12.613396 9999 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 473b2cd51b3342bf8de0165a9a3cf126. Latest consensus state: current_term: 1 leader_uuid: "473b2cd51b3342bf8de0165a9a3cf126" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473b2cd51b3342bf8de0165a9a3cf126" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38839 } } }
I20260811 22:11:12.613551 9998 sys_catalog.cc:455] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "473b2cd51b3342bf8de0165a9a3cf126" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "473b2cd51b3342bf8de0165a9a3cf126" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38839 } } }
I20260811 22:11:12.613688 9998 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:12.613562 9999 sys_catalog.cc:458] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126 [sys.catalog]: This master's current role is: LEADER
I20260811 22:11:12.614544 10008 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:11:12.620596 10008 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:11:12.621316 9991 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:11:12.640781 10008 catalog_manager.cc:1347] Generated new cluster ID: cba3d81cc5d444949cf59efbd0b04dc9
I20260811 22:11:12.641040 10008 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:11:12.657663 10008 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:11:12.661695 10008 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:11:12.687713 10008 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126: Generated new TSK 0
I20260811 22:11:12.689033 10008 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:11:13.806377 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.47.148:42023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38839
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:14.028587 10047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:14.028964 10047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:14.029062 10047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:14.033758 10047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:14.033871 10047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:14.033970 10047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:11:14.040628 10047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.28.47.148:42023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:11:14.042101 10047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:14.043291 10047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:14.058643 10056 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:14.058573 10053 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:14.059033 10052 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:14.059751 10047 server_base.cc:1034] running on GCE node
I20260811 22:11:14.060257 10047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:14.061095 10047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:14.062366 10047 hybrid_clock.cc:648] HybridClock initialized: now 1786486274062315 us; error 70 us; skew 500 ppm
Aug 11 22:11:14 dist-test-slave-vxj2 krb5kdc[9379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486274, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:14.067759 10047 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:11:14.069466 10047 webserver.cc:460] Webserver started at http://127.28.47.129:34009/ using document root <none> and password file <none>
I20260811 22:11:14.070506 10047 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:14.070639 10047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:14.071031 10047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:11:14 dist-test-slave-vxj2 krb5kdc[9379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486274, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:11:14.094602 10047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "819c8a91c4ce428391c51f130f25833a"
format_stamp: "Formatted at 2026-08-11 22:11:14 on dist-test-slave-vxj2"
server_key: "45538093cddd12e921e5fe61150d4fcb"
server_key_iv: "5685ead3141034186e19dedab8c2c3ae"
server_key_version: "kuduclusterkey@0"
I20260811 22:11:14.096809 10047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "819c8a91c4ce428391c51f130f25833a"
format_stamp: "Formatted at 2026-08-11 22:11:14 on dist-test-slave-vxj2"
server_key: "45538093cddd12e921e5fe61150d4fcb"
server_key_iv: "5685ead3141034186e19dedab8c2c3ae"
server_key_version: "kuduclusterkey@0"
I20260811 22:11:14.103591 10047 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.002s
I20260811 22:11:14.134044 10068 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:14.136377 10047 fs_manager.cc:725] Time spent opening block manager: real 0.012s user 0.004s sys 0.000s
I20260811 22:11:14.136632 10047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "819c8a91c4ce428391c51f130f25833a"
format_stamp: "Formatted at 2026-08-11 22:11:14 on dist-test-slave-vxj2"
server_key: "45538093cddd12e921e5fe61150d4fcb"
server_key_iv: "5685ead3141034186e19dedab8c2c3ae"
server_key_version: "kuduclusterkey@0"
I20260811 22:11:14.136843 10047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:14.163144 10047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:14.168660 10047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:14.169029 10047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:14.170419 10047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:14.171788 10047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:14.171867 10047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:14.171929 10047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:14.171959 10047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:14.198238 10047 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:33689
I20260811 22:11:14.200186 10047 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:11:14.200888 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 10047
I20260811 22:11:14.201053 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:11:14.201661 28862 external_mini_cluster.cc:1428] Setting key 6f79aab9e7f738c30bcfd44b3f2765e1
I20260811 22:11:14.213112 10211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:33689 every 8 connection(s)
Aug 11 22:11:14 dist-test-slave-vxj2 krb5kdc[9379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486274, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:11:14.238193 10215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:14.213144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:50203 (local address 127.28.47.190:38839)
0811 22:11:14.213865 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:14.213917 (+ 52us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:14.218145 (+ 4228us) server_negotiation.cc:407] Connection header received
0811 22:11:14.218237 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:14.218241 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:14.218340 (+ 99us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:14.218774 (+ 434us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:14.222074 (+ 3300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:14.223249 (+ 1175us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:14.225610 (+ 2361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:14.226056 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:14.226165 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:14.230746 (+ 4581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:14.230752 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:14.230756 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:14.230816 (+ 60us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:14.233962 (+ 3146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:14.235123 (+ 1161us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:14.235128 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:14.235130 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:14.235248 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:14.235744 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:14.235748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:14.235750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:14.236399 (+ 649us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:14.236553 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:14.237654 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0811 22:11:14.237869 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":530,"thread_start_us":259,"threads_started":1}
I20260811 22:11:14.242089 10216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:14.213877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38839 (local address 127.28.47.129:50203)
0811 22:11:14.215593 (+ 1716us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:14.215669 (+ 76us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:14.217673 (+ 2004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:14.219526 (+ 1853us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:14.219542 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:14.220339 (+ 797us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:14.221741 (+ 1402us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:14.221773 (+ 32us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:14.223573 (+ 1800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:14.223577 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:14.225384 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:14.225406 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:14.226536 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:14.226544 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:14.226889 (+ 345us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:14.227774 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:14.227798 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:14.230401 (+ 2603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:14.234261 (+ 3860us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:14.234271 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:14.234296 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:14.234972 (+ 676us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:14.235411 (+ 439us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:14.235414 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:14.235416 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:14.235610 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:14.236631 (+ 1021us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:14.236650 (+ 19us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:14.237206 (+ 556us) client_negotiation.cc:769] Sending connection context
0811 22:11:14.237544 (+ 338us) client_negotiation.cc:240] Negotiation successful
0811 22:11:14.238090 (+ 546us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1417,"thread_start_us":275,"threads_started":1}
I20260811 22:11:14.244189 10212 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38839
I20260811 22:11:14.244557 10212 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:14.246176 10212 heartbeater.cc:511] Master 127.28.47.190:38839 requested a full tablet report, sending...
I20260811 22:11:14.249123 9950 ts_manager.cc:194] Registered new tserver with Master: 819c8a91c4ce428391c51f130f25833a (127.28.47.129:33689)
I20260811 22:11:14.252681 9950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:50203
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:11:14.272817 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.47.148:42023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38839
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:11:14 dist-test-slave-vxj2 krb5kdc[9379](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486274, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 22:11:14.476584 10220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:14.476981 10220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:14.477088 10220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:14.483489 10220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:14.483628 10220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:14.483752 10220 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:11:14.493368 10220 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.28.47.148:42023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:11:14.494750 10220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:14.496038 10220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:14.509848 10226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:14.510350 10227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:14.511237 10220 server_base.cc:1034] running on GCE node
W20260811 22:11:14.512049 10229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:14.512720 10220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:14.513774 10220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:14.514976 10220 hybrid_clock.cc:648] HybridClock initialized: now 1786486274514935 us; error 56 us; skew 500 ppm
Aug 11 22:11:14 dist-test-slave-vxj2 krb5kdc[9379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486274, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:14.521099 10220 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:11:14.522538 10220 webserver.cc:460] Webserver started at http://127.28.47.130:45811/ using document root <none> and password file <none>
I20260811 22:11:14.523734 10220 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:14.523864 10220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:14.524180 10220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:11:14 dist-test-slave-vxj2 krb5kdc[9379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486274, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:11:14.545596 10220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "8069dbcb96d3432288ae304b105804fc"
format_stamp: "Formatted at 2026-08-11 22:11:14 on dist-test-slave-vxj2"
server_key: "26da74c90fc978379422099d7a241a91"
server_key_iv: "00c098c5b6115d5adb9c308e414846b8"
server_key_version: "kuduclusterkey@0"
I20260811 22:11:14.547586 10220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "8069dbcb96d3432288ae304b105804fc"
format_stamp: "Formatted at 2026-08-11 22:11:14 on dist-test-slave-vxj2"
server_key: "26da74c90fc978379422099d7a241a91"
server_key_iv: "00c098c5b6115d5adb9c308e414846b8"
server_key_version: "kuduclusterkey@0"
I20260811 22:11:14.558493 10220 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.005s sys 0.000s
I20260811 22:11:14.583722 10236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:14.585608 10220 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.006s sys 0.000s
I20260811 22:11:14.585815 10220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "8069dbcb96d3432288ae304b105804fc"
format_stamp: "Formatted at 2026-08-11 22:11:14 on dist-test-slave-vxj2"
server_key: "26da74c90fc978379422099d7a241a91"
server_key_iv: "00c098c5b6115d5adb9c308e414846b8"
server_key_version: "kuduclusterkey@0"
I20260811 22:11:14.586004 10220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:14.604120 10220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:14.609241 10220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:14.609526 10220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:14.610566 10220 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:14.612112 10220 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:14.612202 10220 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:14.612293 10220 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:14.612346 10220 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:14.636545 10220 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:34999
I20260811 22:11:14.638422 10220 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:11:14.640646 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 10220
I20260811 22:11:14.640841 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:11:14.641297 28862 external_mini_cluster.cc:1428] Setting key 0cf05ee325e3521dbe0823b7500e30bb
I20260811 22:11:14.648105 10369 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:34999 every 8 connection(s)
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
Aug 11 22:11:14 dist-test-slave-vxj2 krb5kdc[9379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486274, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:11:14.679203 10375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:14.660757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38839 (local address 127.28.47.130:57733)
0811 22:11:14.661548 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:14.661610 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:14.662993 (+ 1383us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:14.663612 (+ 619us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:14.663620 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:14.664005 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:14.664668 (+ 663us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:14.664685 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:14.665742 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:14.665747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:14.667485 (+ 1738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:14.667502 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:14.668473 (+ 971us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:14.668475 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:14.668695 (+ 220us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:14.669468 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:14.669495 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:14.671546 (+ 2051us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:14.674402 (+ 2856us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:14.674411 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:14.674436 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:14.674947 (+ 511us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:14.675385 (+ 438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:14.675391 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:14.675395 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:14.675632 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:14.676411 (+ 779us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:14.676420 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:14.676868 (+ 448us) client_negotiation.cc:769] Sending connection context
0811 22:11:14.677130 (+ 262us) client_negotiation.cc:240] Negotiation successful
0811 22:11:14.677664 (+ 534us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":514,"thread_start_us":280,"threads_started":1}
I20260811 22:11:14.680223 10215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:14.662195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:57733 (local address 127.28.47.190:38839)
0811 22:11:14.662345 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:14.662350 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:14.662369 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:11:14.663252 (+ 883us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:14.663255 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:14.663318 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:14.663417 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:14.664854 (+ 1437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:14.665550 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:14.667655 (+ 2105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:14.668111 (+ 456us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:14.668214 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:14.671803 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:14.671810 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:14.671816 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:14.671856 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:14.674178 (+ 2322us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:14.675109 (+ 931us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:14.675115 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:14.675119 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:14.675236 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:14.675817 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:14.675821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:14.675823 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:14.676147 (+ 324us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:14.676310 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:14.679790 (+ 3480us) server_negotiation.cc:299] Negotiation successful
0811 22:11:14.680020 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260811 22:11:14.682178 10371 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38839
I20260811 22:11:14.682507 10371 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:14.683272 10371 heartbeater.cc:511] Master 127.28.47.190:38839 requested a full tablet report, sending...
I20260811 22:11:14.684537 9950 ts_manager.cc:194] Registered new tserver with Master: 8069dbcb96d3432288ae304b105804fc (127.28.47.130:34999)
I20260811 22:11:14.685257 9950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:57733
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:11:14.694751 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.47.148:42023/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38839
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39765
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:11:14.912256 10377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:11:14.912724 10377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:11:14.912833 10377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:11:14.919217 10377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:11:14.919432 10377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:11:14.919572 10377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:11:14.928171 10377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39765
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.28.47.148:42023/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38839
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:11:14.929998 10377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:11:14.931572 10377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:11:14.947683 10383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:14.948573 10382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:11:14.949335 10385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:11:14.950596 10377 server_base.cc:1034] running on GCE node
I20260811 22:11:14.951198 10377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:11:14.952116 10377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:11:14.953409 10377 hybrid_clock.cc:648] HybridClock initialized: now 1786486274953374 us; error 56 us; skew 500 ppm
Aug 11 22:11:14 dist-test-slave-vxj2 krb5kdc[9379](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486274, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:11:14.958364 10377 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:11:14.959709 10377 webserver.cc:460] Webserver started at http://127.28.47.131:40507/ using document root <none> and password file <none>
I20260811 22:11:14.960703 10377 fs_manager.cc:359] Metadata directory not provided
I20260811 22:11:14.960968 10377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:11:14.961349 10377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:11:14 dist-test-slave-vxj2 krb5kdc[9379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486274, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:11:14.983493 10377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "c4073418f3564c2d8968d5da1d811dbe"
format_stamp: "Formatted at 2026-08-11 22:11:14 on dist-test-slave-vxj2"
server_key: "6287de70b94e2497645a0d2890a12839"
server_key_iv: "70a214f247574cd676a74c1139e7fcf9"
server_key_version: "kuduclusterkey@0"
I20260811 22:11:14.984602 10377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "c4073418f3564c2d8968d5da1d811dbe"
format_stamp: "Formatted at 2026-08-11 22:11:14 on dist-test-slave-vxj2"
server_key: "6287de70b94e2497645a0d2890a12839"
server_key_iv: "70a214f247574cd676a74c1139e7fcf9"
server_key_version: "kuduclusterkey@0"
I20260811 22:11:14.991168 10377 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.002s sys 0.003s
I20260811 22:11:15.022032 10392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:15.024096 10377 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.004s sys 0.000s
I20260811 22:11:15.024276 10377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "c4073418f3564c2d8968d5da1d811dbe"
format_stamp: "Formatted at 2026-08-11 22:11:14 on dist-test-slave-vxj2"
server_key: "6287de70b94e2497645a0d2890a12839"
server_key_iv: "70a214f247574cd676a74c1139e7fcf9"
server_key_version: "kuduclusterkey@0"
I20260811 22:11:15.024451 10377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:11:15.069876 10377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:11:15.074469 10377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:11:15.074792 10377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:11:15.075758 10377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:11:15.077075 10377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:11:15.077219 10377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:15.077329 10377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:11:15.077399 10377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:11:15.105039 10377 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:45061
I20260811 22:11:15.107280 10377 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:11:15.114382 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 10377
I20260811 22:11:15.114545 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:11:15.114938 28862 external_mini_cluster.cc:1428] Setting key 48adf45a93640ebd4e702702ba8b0213
I20260811 22:11:15.110082 10537 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:45061 every 8 connection(s)
Aug 11 22:11:15 dist-test-slave-vxj2 krb5kdc[9379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486274, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:11:15.137480 10215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.113376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:51633 (local address 127.28.47.190:38839)
0811 22:11:15.113527 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:15.113533 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:15.117891 (+ 4358us) server_negotiation.cc:407] Connection header received
0811 22:11:15.119690 (+ 1799us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:15.119694 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:15.119758 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:15.119810 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:15.122214 (+ 2404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.123060 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.124988 (+ 1928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.125389 (+ 401us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.125484 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.131999 (+ 6515us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:15.132004 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:15.132007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:15.132045 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:15.134280 (+ 2235us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:15.135171 (+ 891us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:15.135175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:15.135177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:15.135295 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:15.135861 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:15.135864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:15.135866 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:15.136116 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:15.136200 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:15.137102 (+ 902us) server_negotiation.cc:299] Negotiation successful
0811 22:11:15.137288 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260811 22:11:15.139644 10540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.112799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38839 (local address 127.28.47.131:51633)
0811 22:11:15.117689 (+ 4890us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:15.117786 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:15.119391 (+ 1605us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:15.120324 (+ 933us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:15.120335 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:15.120960 (+ 625us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:11:15.121976 (+ 1016us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:15.122005 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.123256 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.123260 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:15.124812 (+ 1552us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:15.124828 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.125939 (+ 1111us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.125943 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:15.126176 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.127049 (+ 873us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:11:15.127073 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:11:15.131701 (+ 4628us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:11:15.134560 (+ 2859us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:15.134569 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:15.134591 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:15.135036 (+ 445us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:15.135451 (+ 415us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:11:15.135454 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:11:15.135457 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:11:15.135745 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:11:15.136274 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:11:15.136281 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:11:15.136784 (+ 503us) client_negotiation.cc:769] Sending connection context
0811 22:11:15.137991 (+ 1207us) client_negotiation.cc:240] Negotiation successful
0811 22:11:15.138428 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2388,"thread_start_us":557,"threads_started":1}
I20260811 22:11:15.141834 10538 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38839
I20260811 22:11:15.142120 10538 heartbeater.cc:464] Registering TS with master...
I20260811 22:11:15.142796 10538 heartbeater.cc:511] Master 127.28.47.190:38839 requested a full tablet report, sending...
I20260811 22:11:15.144155 9950 ts_manager.cc:194] Registered new tserver with Master: c4073418f3564c2d8968d5da1d811dbe (127.28.47.131:45061)
I20260811 22:11:15.145350 9950 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:51633
I20260811 22:11:15.154704 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:11:15.169559 10215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.157651 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41766 (local address 127.28.47.190:38839)
0811 22:11:15.157814 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:15.157819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:15.158623 (+ 804us) server_negotiation.cc:407] Connection header received
0811 22:11:15.158858 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:15.158862 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:15.158922 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:15.158979 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:11:15.160222 (+ 1243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.161040 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.162576 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.162928 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.163032 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.165441 (+ 2409us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:11:15.165445 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:11:15.165448 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:11:15.165490 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:11:15.167378 (+ 1888us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:15.167963 (+ 585us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:15.167968 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:15.167969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:15.168071 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:11:15.168424 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:11:15.168427 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:11:15.168429 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:11:15.168673 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:11:15.168789 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:15.169191 (+ 402us) server_negotiation.cc:299] Negotiation successful
0811 22:11:15.169348 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:11:15.174104 9950 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:11:15.179896 9950 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
Aug 11 22:11:15 dist-test-slave-vxj2 krb5kdc[9379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486274, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:11:15.233642 10553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.195118 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:34999 (local address 127.0.0.1:55188)
0811 22:11:15.203092 (+ 7974us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:15.203169 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:15.203325 (+ 156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:15.209477 (+ 6152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:15.209480 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:15.209492 (+ 12us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:15.209627 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:15.209633 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.224764 (+ 15131us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.224768 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:15.226079 (+ 1311us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:15.226097 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.232906 (+ 6809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.232911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:15.233105 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.233179 (+ 74us) client_negotiation.cc:769] Sending connection context
0811 22:11:15.233309 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 22:11:15.233466 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7742,"thread_start_us":217,"threads_started":1}
I20260811 22:11:15.236685 10551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.199959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:33689 (local address 127.0.0.1:35926)
0811 22:11:15.207513 (+ 7554us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:15.207527 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:15.207628 (+ 101us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:15.209254 (+ 1626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:15.209257 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:15.209278 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:15.209385 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:15.209391 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.211354 (+ 1963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.211358 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:15.227860 (+ 16502us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:15.227870 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.234958 (+ 7088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.234962 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:15.235094 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.235121 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:11:15.236403 (+ 1282us) client_negotiation.cc:240] Negotiation successful
0811 22:11:15.236512 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4522,"thread_start_us":163,"threads_started":1}
I20260811 22:11:15.238597 10556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.198550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:45061 (local address 127.0.0.1:37840)
0811 22:11:15.203960 (+ 5410us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:15.203975 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:15.204087 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:15.207189 (+ 3102us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:15.207196 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:15.207223 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:15.207377 (+ 154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:15.207384 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.221505 (+ 14121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.221509 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:15.226975 (+ 5466us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:15.226986 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.238142 (+ 11156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.238146 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:15.238254 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.238290 (+ 36us) client_negotiation.cc:769] Sending connection context
0811 22:11:15.238367 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 22:11:15.238447 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5342,"thread_start_us":131,"threads_started":1}
I20260811 22:11:15.239552 10557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.201430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35926 (local address 127.28.47.129:33689)
0811 22:11:15.204334 (+ 2904us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:15.204341 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:15.207735 (+ 3394us) server_negotiation.cc:407] Connection header received
0811 22:11:15.207836 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:15.207841 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:15.209049 (+ 1208us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:15.209804 (+ 755us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:15.209975 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.211200 (+ 1225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.229378 (+ 18178us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.234561 (+ 5183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.234681 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.234846 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:15.239182 (+ 4336us) server_negotiation.cc:299] Negotiation successful
0811 22:11:15.239347 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2790,"thread_start_us":102,"threads_started":1}
I20260811 22:11:15.240188 10552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.196366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55188 (local address 127.28.47.130:34999)
0811 22:11:15.205722 (+ 9356us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:15.205729 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:15.205759 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:11:15.205862 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:15.205868 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:15.206131 (+ 263us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:15.206301 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:15.223297 (+ 16996us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.224583 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.230609 (+ 6026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.232483 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.232640 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.232820 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:15.239885 (+ 7065us) server_negotiation.cc:299] Negotiation successful
0811 22:11:15.240028 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9237,"thread_start_us":182,"threads_started":1}
I20260811 22:11:15.240890 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.199135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37840 (local address 127.28.47.131:45061)
0811 22:11:15.203763 (+ 4628us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:15.203788 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:15.206516 (+ 2728us) server_negotiation.cc:407] Connection header received
0811 22:11:15.206612 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:15.206618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:15.206808 (+ 190us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:15.206993 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:15.219879 (+ 12886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.221285 (+ 1406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.230836 (+ 9551us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.237769 (+ 6933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.237897 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.238067 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:15.240528 (+ 2461us) server_negotiation.cc:299] Negotiation successful
0811 22:11:15.240672 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4487,"thread_start_us":151,"threads_started":1}
I20260811 22:11:15.245045 10134 tablet_service.cc:1467] Processing CreateTablet for tablet d9408b6ce95d4b02a6a30900dd1fa42d (DEFAULT_TABLE table=test-table [id=8309a31c82ce48e9bdd34ec15cb80872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:15.245971 10290 tablet_service.cc:1467] Processing CreateTablet for tablet d9408b6ce95d4b02a6a30900dd1fa42d (DEFAULT_TABLE table=test-table [id=8309a31c82ce48e9bdd34ec15cb80872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:15.246778 10134 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9408b6ce95d4b02a6a30900dd1fa42d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:15.247252 10290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9408b6ce95d4b02a6a30900dd1fa42d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:15.248068 10446 tablet_service.cc:1467] Processing CreateTablet for tablet d9408b6ce95d4b02a6a30900dd1fa42d (DEFAULT_TABLE table=test-table [id=8309a31c82ce48e9bdd34ec15cb80872]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:11:15.249347 10446 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d9408b6ce95d4b02a6a30900dd1fa42d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:11:15.257265 10212 heartbeater.cc:503] Master 127.28.47.190:38839 was elected leader, sending a full tablet report...
I20260811 22:11:15.265046 10558 tablet_bootstrap.cc:492] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc: Bootstrap starting.
I20260811 22:11:15.265445 10560 tablet_bootstrap.cc:492] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a: Bootstrap starting.
I20260811 22:11:15.270491 10560 tablet_bootstrap.cc:654] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:15.272257 10560 log.cc:824] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:15.275329 10560 tablet_bootstrap.cc:492] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a: No bootstrap required, opened a new log
I20260811 22:11:15.275611 10560 ts_tablet_manager.cc:1397] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a: Time spent bootstrapping tablet: real 0.010s user 0.008s sys 0.000s
I20260811 22:11:15.282008 10559 tablet_bootstrap.cc:492] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe: Bootstrap starting.
I20260811 22:11:15.283912 10560 raft_consensus.cc:348] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8069dbcb96d3432288ae304b105804fc" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 34999 } } peers { permanent_uuid: "819c8a91c4ce428391c51f130f25833a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33689 } } peers { permanent_uuid: "c4073418f3564c2d8968d5da1d811dbe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45061 } }
I20260811 22:11:15.285854 10560 raft_consensus.cc:374] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:15.286016 10560 raft_consensus.cc:729] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 819c8a91c4ce428391c51f130f25833a, State: Initialized, Role: FOLLOWER
I20260811 22:11:15.286314 10559 tablet_bootstrap.cc:654] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:15.287123 10560 consensus_queue.cc:260] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8069dbcb96d3432288ae304b105804fc" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 34999 } } peers { permanent_uuid: "819c8a91c4ce428391c51f130f25833a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33689 } } peers { permanent_uuid: "c4073418f3564c2d8968d5da1d811dbe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45061 } }
I20260811 22:11:15.288403 10559 log.cc:824] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:15.289399 10560 ts_tablet_manager.cc:1428] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a: Time spent starting tablet: real 0.014s user 0.005s sys 0.006s
I20260811 22:11:15.295790 10559 tablet_bootstrap.cc:492] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe: No bootstrap required, opened a new log
I20260811 22:11:15.296221 10559 ts_tablet_manager.cc:1397] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.003s
I20260811 22:11:15.296762 10558 tablet_bootstrap.cc:654] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc: Neither blocks nor log segments found. Creating new log.
I20260811 22:11:15.298347 10558 log.cc:824] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc: Log is configured to *not* fsync() on all Append() calls
I20260811 22:11:15.314786 10559 raft_consensus.cc:348] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8069dbcb96d3432288ae304b105804fc" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 34999 } } peers { permanent_uuid: "819c8a91c4ce428391c51f130f25833a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33689 } } peers { permanent_uuid: "c4073418f3564c2d8968d5da1d811dbe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45061 } }
I20260811 22:11:15.315358 10559 raft_consensus.cc:374] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:15.315474 10559 raft_consensus.cc:729] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c4073418f3564c2d8968d5da1d811dbe, State: Initialized, Role: FOLLOWER
I20260811 22:11:15.316221 10559 consensus_queue.cc:260] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8069dbcb96d3432288ae304b105804fc" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 34999 } } peers { permanent_uuid: "819c8a91c4ce428391c51f130f25833a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33689 } } peers { permanent_uuid: "c4073418f3564c2d8968d5da1d811dbe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45061 } }
I20260811 22:11:15.317865 10559 ts_tablet_manager.cc:1428] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe: Time spent starting tablet: real 0.021s user 0.001s sys 0.009s
I20260811 22:11:15.318358 10538 heartbeater.cc:503] Master 127.28.47.190:38839 was elected leader, sending a full tablet report...
I20260811 22:11:15.323328 10558 tablet_bootstrap.cc:492] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc: No bootstrap required, opened a new log
I20260811 22:11:15.325109 10558 ts_tablet_manager.cc:1397] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc: Time spent bootstrapping tablet: real 0.060s user 0.003s sys 0.006s
I20260811 22:11:15.333125 10558 raft_consensus.cc:348] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8069dbcb96d3432288ae304b105804fc" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 34999 } } peers { permanent_uuid: "819c8a91c4ce428391c51f130f25833a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33689 } } peers { permanent_uuid: "c4073418f3564c2d8968d5da1d811dbe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45061 } }
I20260811 22:11:15.333693 10558 raft_consensus.cc:374] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:11:15.333812 10558 raft_consensus.cc:729] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8069dbcb96d3432288ae304b105804fc, State: Initialized, Role: FOLLOWER
I20260811 22:11:15.334934 10558 consensus_queue.cc:260] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8069dbcb96d3432288ae304b105804fc" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 34999 } } peers { permanent_uuid: "819c8a91c4ce428391c51f130f25833a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33689 } } peers { permanent_uuid: "c4073418f3564c2d8968d5da1d811dbe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45061 } }
I20260811 22:11:15.336696 10558 ts_tablet_manager.cc:1428] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc: Time spent starting tablet: real 0.011s user 0.003s sys 0.006s
I20260811 22:11:15.337226 10371 heartbeater.cc:503] Master 127.28.47.190:38839 was elected leader, sending a full tablet report...
W20260811 22:11:15.365527 10539 tablet.cc:2369] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:11:15.411829 10373 tablet.cc:2369] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:11:15.462478 10213 tablet.cc:2369] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 11 22:11:15 dist-test-slave-vxj2 krb5kdc[9379](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486274, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:11:15.657114 10562 raft_consensus.cc:482] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:11:15.657433 10562 raft_consensus.cc:504] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8069dbcb96d3432288ae304b105804fc" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 34999 } } peers { permanent_uuid: "819c8a91c4ce428391c51f130f25833a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33689 } } peers { permanent_uuid: "c4073418f3564c2d8968d5da1d811dbe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45061 } }
I20260811 22:11:15.659060 10562 leader_election.cc:288] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8069dbcb96d3432288ae304b105804fc (127.28.47.130:34999), c4073418f3564c2d8968d5da1d811dbe (127.28.47.131:45061)
I20260811 22:11:15.667804 10552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.660914 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:57363 (local address 127.28.47.130:34999)
0811 22:11:15.661055 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:15.661060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:15.661076 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:11:15.661133 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:15.661136 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:15.661215 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:15.661296 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:15.662032 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.662791 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.665984 (+ 3193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.666642 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.666744 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.666802 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:15.667558 (+ 756us) server_negotiation.cc:299] Negotiation successful
0811 22:11:15.667663 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:11:15.669199 10570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.659538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:34999 (local address 127.28.47.129:57363)
0811 22:11:15.660083 (+ 545us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:15.660102 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:15.660246 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:15.661427 (+ 1181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:15.661433 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:15.661458 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:15.661890 (+ 432us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:15.661898 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.663572 (+ 1674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.663576 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:15.665805 (+ 2229us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:15.665826 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.667002 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.667005 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:15.667170 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.667195 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:11:15.668690 (+ 1495us) client_negotiation.cc:240] Negotiation successful
0811 22:11:15.668794 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":447,"thread_start_us":210,"threads_started":1}
I20260811 22:11:15.670910 10305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9408b6ce95d4b02a6a30900dd1fa42d" candidate_uuid: "819c8a91c4ce428391c51f130f25833a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8069dbcb96d3432288ae304b105804fc" is_pre_election: true
I20260811 22:11:15.671000 10555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.661098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:44457 (local address 127.28.47.131:45061)
0811 22:11:15.661256 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:15.661262 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:15.661279 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:11:15.661343 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:15.661345 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:15.661395 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:15.661466 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:11:15.663501 (+ 2035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.664415 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.669306 (+ 4891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.669962 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.670082 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.670141 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:15.670772 (+ 631us) server_negotiation.cc:299] Negotiation successful
0811 22:11:15.670876 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:11:15.671072 10571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.660190 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:45061 (local address 127.28.47.129:44457)
0811 22:11:15.660623 (+ 433us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:15.660641 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:11:15.660772 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:11:15.662958 (+ 2186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:11:15.662961 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:11:15.662984 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:11:15.663324 (+ 340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:15.663334 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.664572 (+ 1238us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.664575 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:15.666078 (+ 1503us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:11:15.666087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.670418 (+ 4331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.670422 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:11:15.670619 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.670642 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:11:15.670746 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 22:11:15.670838 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"thread_start_us":156,"threads_started":1}
I20260811 22:11:15.671356 10305 raft_consensus.cc:2454] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 819c8a91c4ce428391c51f130f25833a in term 0.
I20260811 22:11:15.671921 10474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9408b6ce95d4b02a6a30900dd1fa42d" candidate_uuid: "819c8a91c4ce428391c51f130f25833a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4073418f3564c2d8968d5da1d811dbe" is_pre_election: true
I20260811 22:11:15.672123 10073 leader_election.cc:302] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8069dbcb96d3432288ae304b105804fc, 819c8a91c4ce428391c51f130f25833a; no voters:
I20260811 22:11:15.672253 10474 raft_consensus.cc:2454] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 819c8a91c4ce428391c51f130f25833a in term 0.
I20260811 22:11:15.672497 10562 raft_consensus.cc:2781] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:11:15.672617 10562 raft_consensus.cc:482] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:11:15.672685 10562 raft_consensus.cc:3037] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:15.675019 10562 raft_consensus.cc:504] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8069dbcb96d3432288ae304b105804fc" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 34999 } } peers { permanent_uuid: "819c8a91c4ce428391c51f130f25833a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33689 } } peers { permanent_uuid: "c4073418f3564c2d8968d5da1d811dbe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45061 } }
I20260811 22:11:15.675555 10562 leader_election.cc:288] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [CANDIDATE]: Term 1 election: Requested vote from peers 8069dbcb96d3432288ae304b105804fc (127.28.47.130:34999), c4073418f3564c2d8968d5da1d811dbe (127.28.47.131:45061)
I20260811 22:11:15.675966 10305 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9408b6ce95d4b02a6a30900dd1fa42d" candidate_uuid: "819c8a91c4ce428391c51f130f25833a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8069dbcb96d3432288ae304b105804fc"
I20260811 22:11:15.676033 10474 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d9408b6ce95d4b02a6a30900dd1fa42d" candidate_uuid: "819c8a91c4ce428391c51f130f25833a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c4073418f3564c2d8968d5da1d811dbe"
I20260811 22:11:15.676137 10305 raft_consensus.cc:3037] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:15.676186 10474 raft_consensus.cc:3037] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:11:15.677608 10305 raft_consensus.cc:2454] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 819c8a91c4ce428391c51f130f25833a in term 1.
I20260811 22:11:15.677826 10474 raft_consensus.cc:2454] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 819c8a91c4ce428391c51f130f25833a in term 1.
I20260811 22:11:15.678033 10073 leader_election.cc:302] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8069dbcb96d3432288ae304b105804fc, 819c8a91c4ce428391c51f130f25833a; no voters:
I20260811 22:11:15.678256 10562 raft_consensus.cc:2781] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:11:15.678544 10562 raft_consensus.cc:686] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [term 1 LEADER]: Becoming Leader. State: Replica: 819c8a91c4ce428391c51f130f25833a, State: Running, Role: LEADER
I20260811 22:11:15.679020 10562 consensus_queue.cc:237] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8069dbcb96d3432288ae304b105804fc" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 34999 } } peers { permanent_uuid: "819c8a91c4ce428391c51f130f25833a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33689 } } peers { permanent_uuid: "c4073418f3564c2d8968d5da1d811dbe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45061 } }
I20260811 22:11:15.683475 9950 catalog_manager.cc:5495] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a reported cstate change: term changed from 0 to 1, leader changed from <none> to 819c8a91c4ce428391c51f130f25833a (127.28.47.129). New cstate: current_term: 1 leader_uuid: "819c8a91c4ce428391c51f130f25833a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8069dbcb96d3432288ae304b105804fc" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 34999 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "819c8a91c4ce428391c51f130f25833a" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33689 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "c4073418f3564c2d8968d5da1d811dbe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45061 } health_report { overall_health: UNKNOWN } } }
I20260811 22:11:15.783998 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.776418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35930 (local address 127.28.47.129:33689)
0811 22:11:15.777388 (+ 970us) server_negotiation.cc:206] Beginning negotiation
0811 22:11:15.777394 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:11:15.777411 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:11:15.777532 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:11:15.777536 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:11:15.777616 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:11:15.778116 (+ 500us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:11:15.778235 (+ 119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.779057 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.780674 (+ 1617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:11:15.780956 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:11:15.781505 (+ 549us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:11:15.781678 (+ 173us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:11:15.782327 (+ 649us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:11:15.783393 (+ 1066us) server_negotiation.cc:1035] Waiting for connection context
0811 22:11:15.783437 (+ 44us) server_negotiation.cc:299] Negotiation successful
0811 22:11:15.783518 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":873,"thread_start_us":74,"threads_started":1}
I20260811 22:11:15.801529 10474 raft_consensus.cc:1261] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe [term 1 FOLLOWER]: Refusing update from remote peer 819c8a91c4ce428391c51f130f25833a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:15.801951 10305 raft_consensus.cc:1261] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc [term 1 FOLLOWER]: Refusing update from remote peer 819c8a91c4ce428391c51f130f25833a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:11:15.802594 10562 consensus_queue.cc:1035] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [LEADER]: Connected to new peer: Peer: permanent_uuid: "c4073418f3564c2d8968d5da1d811dbe" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45061 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:15.803264 10572 consensus_queue.cc:1035] T d9408b6ce95d4b02a6a30900dd1fa42d P 819c8a91c4ce428391c51f130f25833a [LEADER]: Connected to new peer: Peer: permanent_uuid: "8069dbcb96d3432288ae304b105804fc" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 34999 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:11:15.820334 10582 mvcc.cc:205] Tried to move back new op lower bound from 7317447785672138752 to 7317447785185009664. Current Snapshot: MvccSnapshot[applied={T|T < 7317447785672138752}]
I20260811 22:11:15.828748 10580 mvcc.cc:205] Tried to move back new op lower bound from 7317447785672138752 to 7317447785185009664. Current Snapshot: MvccSnapshot[applied={T|T < 7317447785672138752}]
I20260811 22:11:15.835359 9950 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:15.835616 9950 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:11:15.839036 9950 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 473b2cd51b3342bf8de0165a9a3cf126: Sending DeleteTablet for 3 replicas of tablet d9408b6ce95d4b02a6a30900dd1fa42d
I20260811 22:11:15.840222 10290 tablet_service.cc:1514] Processing DeleteTablet for tablet d9408b6ce95d4b02a6a30900dd1fa42d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:15 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:55188
I20260811 22:11:15.841662 10134 tablet_service.cc:1514] Processing DeleteTablet for tablet d9408b6ce95d4b02a6a30900dd1fa42d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:15 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:35926
I20260811 22:11:15.842964 10446 tablet_service.cc:1514] Processing DeleteTablet for tablet d9408b6ce95d4b02a6a30900dd1fa42d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:11:15 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:37840
I20260811 22:11:15.844609 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 10047
I20260811 22:11:15.853775 10591 tablet_replica.cc:330] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe: stopping tablet replica
I20260811 22:11:15.854286 10591 raft_consensus.cc:2229] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:15.854775 10591 raft_consensus.cc:2258] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:15.858404 10589 tablet_replica.cc:330] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc: stopping tablet replica
I20260811 22:11:15.858904 10589 raft_consensus.cc:2229] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:11:15.859360 10589 raft_consensus.cc:2258] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:11:15.861436 10591 ts_tablet_manager.cc:1905] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:15.864892 10591 ts_tablet_manager.cc:1918] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:11:15.865056 10591 log.cc:1196] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/d9408b6ce95d4b02a6a30900dd1fa42d
I20260811 22:11:15.872032 10589 ts_tablet_manager.cc:1905] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:11:15.875483 10589 ts_tablet_manager.cc:1918] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:11:15.875651 10589 log.cc:1196] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/d9408b6ce95d4b02a6a30900dd1fa42d
I20260811 22:11:15.876047 10589 ts_tablet_manager.cc:1939] T d9408b6ce95d4b02a6a30900dd1fa42d P 8069dbcb96d3432288ae304b105804fc: Deleting consensus metadata
I20260811 22:11:15.877584 10591 ts_tablet_manager.cc:1939] T d9408b6ce95d4b02a6a30900dd1fa42d P c4073418f3564c2d8968d5da1d811dbe: Deleting consensus metadata
I20260811 22:11:15.879343 9934 catalog_manager.cc:4841] TS c4073418f3564c2d8968d5da1d811dbe (127.28.47.131:45061): tablet d9408b6ce95d4b02a6a30900dd1fa42d (table test-table [id=8309a31c82ce48e9bdd34ec15cb80872]) successfully deleted
W20260811 22:11:15.879738 9935 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:33689 (error 108)
I20260811 22:11:15.880271 9935 catalog_manager.cc:4841] TS 8069dbcb96d3432288ae304b105804fc (127.28.47.130:34999): tablet d9408b6ce95d4b02a6a30900dd1fa42d (table test-table [id=8309a31c82ce48e9bdd34ec15cb80872]) successfully deleted
I20260811 22:11:15.880541 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 10220
I20260811 22:11:15.893710 10597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:11:15.889914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:33689 (local address 127.0.0.1:35934)
0811 22:11:15.893444 (+ 3530us) negotiation.cc:107] Waiting for socket to connect
0811 22:11:15.893568 (+ 124us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:33689: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3414,"thread_start_us":76,"threads_started":1}
W20260811 22:11:15.893956 9935 catalog_manager.cc:4584] TS 819c8a91c4ce428391c51f130f25833a (127.28.47.129:33689): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d9408b6ce95d4b02a6a30900dd1fa42d: Network error: Client connection negotiation failed: client connection to 127.28.47.129:33689: connect: Connection refused (error 111)
I20260811 22:11:15.895434 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 10377
I20260811 22:11:15.909348 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 9914
I20260811 22:11:15.920771 28862 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 22:11:16.268895 28862 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 22:11:16.269124 28862 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:11:16.615520 28862 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:11:16.615 UTC [9414] LOG: received smart shutdown request
2026-08-11 22:11:16.620 UTC [9414] LOG: background worker "logical replication launcher" (PID 9424) exited with exit code 1
2026-08-11 22:11:16.620 UTC [9419] LOG: shutting down
2026-08-11 22:11:16.847 UTC [9414] LOG: database system is shut down
2026-08-11T22:11:16Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (58641 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:11:17 dist-test-slave-vxj2 krb5kdc[10608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:11:17 dist-test-slave-vxj2 krb5kdc[10608](info): set up 2 sockets
Aug 11 22:11:17 dist-test-slave-vxj2 krb5kdc[10608](info): commencing operation
krb5kdc: starting...
W20260811 22:11:18.171028 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.172s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:11:18 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486278, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:11:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:11:18Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.28.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.28.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/rangeradmin_127.28.47.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/rangeradmin_127.28.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.28.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.28.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/rangerlookup_127.28.47.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/rangerlookup_127.28.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.28.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/HTTP_127.28.47.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/HTTP_127.28.47.148@KRBTEST.COM.keytab.
I20260811 22:11:18.380985 28862 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-tasktpwotM/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/postgres -l logfile start
2026-08-11 22:11:21.603 UTC [10645] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-11 22:11:21.604 UTC [10645] LOG: listening on IPv4 address "127.28.47.148", port 38499
2026-08-11 22:11:21.609 UTC [10645] LOG: listening on Unix socket "/tmp/.s.PGSQL.38499"
2026-08-11 22:11:21.634 UTC [10648] LOG: database system was shut down at 2026-08-11 22:11:19 UTC
2026-08-11 22:11:21.639 UTC [10645] LOG: database system is ready to accept connections
I20260811 22:11:22.703138 28862 mini_postgres.cc:96] Postgres bound to 38499
2026-08-11 22:11:22.710 UTC [10656] FATAL: database "slave" does not exist
127.28.47.148:38499 - accepting connections
I20260811 22:11:22.711603 28862 mini_ranger.cc:161] Starting Ranger...
I20260811 22:11:22.730917 28862 mini_ranger.cc:84] Created miniranger Postgres user
I20260811 22:11:23.392746 28862 mini_ranger.cc:87] Created ranger Postgres database
I20260811 22:11:23.392834 28862 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-admin
2026-08-11 22:11:23,715 [I] DB FLAVOR :POSTGRES
2026-08-11 22:11:23,715 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:11:23,716 [I] Checking connection..
2026-08-11 22:11:23,716 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-11 22:11:24,017 [I] Checking connection passed.
2026-08-11 22:11:24,017 [I] --------- Verifying version history table ---------
2026-08-11 22:11:24,017 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:11:24,316 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:11:24,317 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:11:24,609 [I] Table x_db_version_h does not exist in database ranger
2026-08-11 22:11:24,609 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-11 22:11:24,610 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-11 22:11:24.891 UTC [10739] WARNING: there is no transaction in progress
2026-08-11 22:11:24,899 [I] create_dbversion_catalog.sql file imported successfully
2026-08-11 22:11:24,900 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-11 22:11:25,186 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-11 22:11:25,186 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:11:25,186 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-11 22:11:25,446 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-11 22:11:25,732 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-vxj2.c.gcp-upstream.internal','N') ;"
2026-08-11 22:11:26,002 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:11:26,311 [I] Table x_portal_user does not exist in database ranger
2026-08-11 22:11:26,312 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:11:26,588 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-11 22:11:26,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:11:26,879 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:11:27,177 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-11 22:11:27,177 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-11 22:11:28.455 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.477 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.499 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.519 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.540 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.563 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.638 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.647 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.664 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.676 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.689 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.706 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.720 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.731 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.741 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:28.751 UTC [10880] WARNING: there is no transaction in progress
2026-08-11 22:11:29,645 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-11 22:11:29,645 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-11 22:11:29,911 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-11 22:11:29,911 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-11 22:11:30,198 [I] Table x_portal_user already exists in database 'ranger'
2026-08-11 22:11:30,199 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-11 22:11:30,476 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-11 22:11:30,476 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-11 22:11:30,740 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-11 22:11:31,021 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-vxj2.c.gcp-upstream.internal';"
2026-08-11 22:11:31,303 [I] CORE_DB_SCHEMA import status has been updated
2026-08-11 22:11:31,304 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-11 22:11:31,576 [I] DB_PATCHES have already been applied
I20260811 22:11:31.582983 28862 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-admin:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-tasktpwotM/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260811 22:11:31.583065 28862 mini_ranger.cc:193] Using host: 127.28.47.148
I20260811 22:11:31.588150 28862 mini_ranger.cc:227] Ranger admin URL: http://127.28.47.148:53351
Aug 11, 2026 10:11:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:11:31 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:11:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-admin/ews/webapp
Aug 11, 2026 10:11:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 11, 2026 10:11:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 11, 2026 10:11:32 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-admin/ews/webapp].
Aug 11, 2026 10:11:32 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260811 22:11:32.858477 28862 mini_ranger.cc:160] Time spent starting Ranger: real 10.147s user 0.001s sys 0.012s
Aug 11, 2026 10:11:57 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 11, 2026 10:11:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Aug 11, 2026 10:11:57 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 11, 2026 10:11:57 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 11, 2026 10:11:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 11, 2026 10:11:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 11, 2026 10:11:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 11, 2026 10:11:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 11, 2026 10:11:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 11, 2026 10:11:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 11, 2026 10:11:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 11, 2026 10:11:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 11, 2026 10:11:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 11, 2026 10:11:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 11, 2026 10:11:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 11, 2026 10:11:57 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 11, 2026 10:11:57 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 11, 2026 10:11:58 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260811 22:11:58.855580 28862 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.28.47.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.28.47.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/rangerkms_127.28.47.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.28.47.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/rangerkms_127.28.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.28.47.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.28.47.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.28.47.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/HTTP_127.28.47.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.28.47.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/HTTP_127.28.47.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 11 22:11:58 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486318, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260811 22:11:58.961381 28862 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260811 22:11:58.974537 28862 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260811 22:12:00.071396 28862 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260811 22:12:00.071501 28862 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-kms
I20260811 22:12:00.071524 28862 mini_ranger_kms.cc:227] Using postgres at 127.28.47.148:38499
2026-08-11 22:12:00,267 [I] DB FLAVOR :POSTGRES
2026-08-11 22:12:00,267 [I] --------- Verifying Ranger DB connection ---------
2026-08-11 22:12:00,267 [I] Checking connection
2026-08-11 22:12:00,268 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-11 22:12:00,544 [I] connection success
2026-08-11 22:12:00,544 [I] --------- Verifying Ranger DB tables ---------
2026-08-11 22:12:00,544 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-11 22:12:00,545 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-11 22:12:00,821 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-11 22:12:00,821 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-11 22:12:00,822 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-11 22:12:00,822 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.28.47.148:38499/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-11 22:12:01,107 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260811 22:12:01.696585 28862 mini_ranger_kms.cc:310] Created kms service
I20260811 22:12:01.933547 28862 mini_ranger_kms.cc:326] Created kudu user
I20260811 22:12:01.985994 28862 mini_ranger_kms.cc:343] Created rangerkms user
Aug 11 22:12:01 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486318, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:12:02.475917 28862 mini_ranger_kms.cc:384] Added ranger policy
I20260811 22:12:02.476099 28862 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-kms:/tmp/dist-test-tasktpwotM/build/debug/bin/postgresql.jar:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-tasktpwotM/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260811 22:12:02.476122 28862 mini_ranger_kms.cc:240] Using host: 127.28.47.148
I20260811 22:12:02.481722 28862 mini_ranger_kms.cc:276] Ranger KMS PID: 11100
I20260811 22:12:02.481897 28862 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.28.47.148:44293
Aug 11, 2026 10:12:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 11, 2026 10:12:03 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 11, 2026 10:12:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 11, 2026 10:12:03 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 11, 2026 10:12:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-kms/ews/webapp].
Aug 11, 2026 10:12:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 11 22:12:04 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486324, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.47.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:05.098253 28862 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.137s user 0.006s sys 0.173s
I20260811 22:12:05.098398 28862 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260811 22:12:05.098492 28862 mini_ranger_kms.cc:401] 127.28.47.148:44293/kms/v1/keys
Aug 11 22:12:05 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486324, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.47.148@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
Aug 11 22:12:05 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486324, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.28.47.148@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
Aug 11 22:12:08 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:12:08.308802 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:41763
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:41763
--ranger_config_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.47.148:44293/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:08.456084 11163 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:08.456435 11163 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:08.456480 11163 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:08.460980 11163 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:08.461090 11163 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:08.461112 11163 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:08.461129 11163 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:08.461233 11163 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:08.467659 11163 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/wal
--ranger_kms_url=127.28.47.148:44293/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:41763
--ranger_config_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:41763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11163
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:08.469036 11163 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:08.470290 11163 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:08.480559 11169 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:08.480612 11171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:08.480646 11174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:08.480607 11163 server_base.cc:1034] running on GCE node
I20260811 22:12:08.481729 11163 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:08.483250 11163 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:08.486720 11163 hybrid_clock.cc:648] HybridClock initialized: now 1786486328486620 us; error 93 us; skew 500 ppm
Aug 11 22:12:08 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486328, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:08.491261 11163 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:12:08.492483 11163 webserver.cc:460] Webserver started at http://127.28.47.190:40011/ using document root <none> and password file <none>
I20260811 22:12:08.493221 11163 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:08.493323 11163 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:08.493595 11163 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:12:08 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486328, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:12:08.556886 11163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "413c8b4e69b64ef39d7f9a27167aa686"
format_stamp: "Formatted at 2026-08-11 22:12:08 on dist-test-slave-vxj2"
server_key: "7619df2aeafb39e9944645d1f561f547"
server_key_iv: "dcbb0eddc0edb98be876f724d5d12133"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:08.557688 11163 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "413c8b4e69b64ef39d7f9a27167aa686"
format_stamp: "Formatted at 2026-08-11 22:12:08 on dist-test-slave-vxj2"
server_key: "7619df2aeafb39e9944645d1f561f547"
server_key_iv: "dcbb0eddc0edb98be876f724d5d12133"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:08.562222 11163 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:12:08.578976 11183 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:08.580658 11163 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:12:08.580852 11163 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "413c8b4e69b64ef39d7f9a27167aa686"
format_stamp: "Formatted at 2026-08-11 22:12:08 on dist-test-slave-vxj2"
server_key: "7619df2aeafb39e9944645d1f561f547"
server_key_iv: "dcbb0eddc0edb98be876f724d5d12133"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:08.581007 11163 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:08.602813 11163 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:08.606026 11163 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:08.606266 11163 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:08.614162 11163 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:41763
I20260811 22:12:08.614192 11240 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:41763 every 8 connection(s)
I20260811 22:12:08.615331 11163 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:08.618837 11241 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:08.620641 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 11163
I20260811 22:12:08.620781 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:12:08.621138 28862 external_mini_cluster.cc:1428] Setting key 5c33f500c0d113c3be6c6ffbdf4bdf6d
I20260811 22:12:08.626565 11241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686: Bootstrap starting.
I20260811 22:12:08.629529 11241 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:08.630611 11241 log.cc:824] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:12:08 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486328, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:08.633368 11241 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686: No bootstrap required, opened a new log
I20260811 22:12:08.637274 11241 raft_consensus.cc:348] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413c8b4e69b64ef39d7f9a27167aa686" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 41763 } }
I20260811 22:12:08.637583 11241 raft_consensus.cc:374] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:08.637667 11241 raft_consensus.cc:729] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413c8b4e69b64ef39d7f9a27167aa686, State: Initialized, Role: FOLLOWER
I20260811 22:12:08.638199 11241 consensus_queue.cc:260] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413c8b4e69b64ef39d7f9a27167aa686" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 41763 } }
I20260811 22:12:08.638381 11241 raft_consensus.cc:388] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:08.638473 11241 raft_consensus.cc:482] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:08.638613 11241 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:08.638789 11245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:08.622968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46630 (local address 127.28.47.190:41763)
0811 22:12:08.623476 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:08.623494 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:08.623538 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:12:08.624292 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:08.624313 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:08.624717 (+ 404us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:08.625063 (+ 346us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:08.626133 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.627383 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.628621 (+ 1238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:08.629152 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:08.629372 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:08.632629 (+ 3257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:08.632635 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:08.632651 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:08.632680 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:08.635165 (+ 2485us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:08.635704 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:08.635709 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:08.635714 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:08.635832 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:08.636296 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:08.636300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:08.636302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:08.636878 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:08.637137 (+ 259us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:08.637489 (+ 352us) server_negotiation.cc:299] Negotiation successful
0811 22:12:08.637839 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":332,"thread_start_us":166,"threads_started":1}
I20260811 22:12:08.640050 11241 raft_consensus.cc:504] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413c8b4e69b64ef39d7f9a27167aa686" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 41763 } }
I20260811 22:12:08.640475 11241 leader_election.cc:302] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 413c8b4e69b64ef39d7f9a27167aa686; no voters:
I20260811 22:12:08.640894 11241 leader_election.cc:288] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:08.640988 11247 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:08.641245 11247 raft_consensus.cc:686] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 1 LEADER]: Becoming Leader. State: Replica: 413c8b4e69b64ef39d7f9a27167aa686, State: Running, Role: LEADER
I20260811 22:12:08.641985 11247 consensus_queue.cc:237] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413c8b4e69b64ef39d7f9a27167aa686" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 41763 } }
I20260811 22:12:08.642373 11241 sys_catalog.cc:564] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:08.643316 11247 sys_catalog.cc:455] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 413c8b4e69b64ef39d7f9a27167aa686. Latest consensus state: current_term: 1 leader_uuid: "413c8b4e69b64ef39d7f9a27167aa686" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413c8b4e69b64ef39d7f9a27167aa686" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 41763 } } }
I20260811 22:12:08.643546 11247 sys_catalog.cc:458] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:08.644011 11259 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:08.644052 11249 sys_catalog.cc:455] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "413c8b4e69b64ef39d7f9a27167aa686" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413c8b4e69b64ef39d7f9a27167aa686" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 41763 } } }
I20260811 22:12:08.644224 11249 sys_catalog.cc:458] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:08.647434 11259 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:08.647974 11241 ranger_client.cc:318] Using new properties file: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:12:08.658201 11259 catalog_manager.cc:1347] Generated new cluster ID: 2180ad5b1f294969b7c561680166539e
I20260811 22:12:08.658324 11259 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:08.684870 11259 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:08.686390 11259 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:08.703045 11259 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686: Generated new TSK 0
I20260811 22:12:08.704208 11259 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:12:09.472385 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.47.148:44293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:41763
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:09.608887 11297 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:09.609320 11297 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:09.609416 11297 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:09.614935 11297 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:09.615029 11297 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:09.615097 11297 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:12:09.622084 11297 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.28.47.148:44293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:41763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11297
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:09.623718 11297 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:09.625367 11297 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:09.640839 11307 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:09.640867 11308 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:09.641472 11297 server_base.cc:1034] running on GCE node
W20260811 22:12:09.641510 11310 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:09.642125 11297 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:09.643110 11297 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:09.644341 11297 hybrid_clock.cc:648] HybridClock initialized: now 1786486329644282 us; error 77 us; skew 500 ppm
Aug 11 22:12:09 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486329, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:09.649574 11297 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:12:09.650990 11297 webserver.cc:460] Webserver started at http://127.28.47.129:34413/ using document root <none> and password file <none>
I20260811 22:12:09.651757 11297 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:09.651875 11297 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:09.652195 11297 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:12:09 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486329, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:12:09.669353 11297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "cb07dda674f44e7698f0cabffacb3b35"
format_stamp: "Formatted at 2026-08-11 22:12:09 on dist-test-slave-vxj2"
server_key: "86ce2eeeb5352bd9cfd5a17ca687ddef"
server_key_iv: "15a06df09b61979de9d821d938ae060a"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:09.672294 11297 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "cb07dda674f44e7698f0cabffacb3b35"
format_stamp: "Formatted at 2026-08-11 22:12:09 on dist-test-slave-vxj2"
server_key: "86ce2eeeb5352bd9cfd5a17ca687ddef"
server_key_iv: "15a06df09b61979de9d821d938ae060a"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:09.683184 11297 fs_manager.cc:691] Time spent creating directory manager: real 0.010s user 0.002s sys 0.003s
I20260811 22:12:09.700362 11319 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:09.702628 11297 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:12:09.702870 11297 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "cb07dda674f44e7698f0cabffacb3b35"
format_stamp: "Formatted at 2026-08-11 22:12:09 on dist-test-slave-vxj2"
server_key: "86ce2eeeb5352bd9cfd5a17ca687ddef"
server_key_iv: "15a06df09b61979de9d821d938ae060a"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:09.703071 11297 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:09.734961 11297 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:09.746605 11297 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:09.746907 11297 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:09.747864 11297 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:09.749128 11297 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:09.749274 11297 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.001s
I20260811 22:12:09.749378 11297 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:09.749434 11297 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:09.765743 11297 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:36403
I20260811 22:12:09.765792 11464 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:36403 every 8 connection(s)
I20260811 22:12:09.767295 11297 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:09.775808 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 11297
I20260811 22:12:09.775947 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:12:09.776257 28862 external_mini_cluster.cc:1428] Setting key ace404c49f1f01f3e5ff8b568cadf7c5
Aug 11 22:12:09 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486329, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:09.791694 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:09.770023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:41639 (local address 127.28.47.190:41763)
0811 22:12:09.770326 (+ 303us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:09.770337 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:09.771219 (+ 882us) server_negotiation.cc:407] Connection header received
0811 22:12:09.772446 (+ 1227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:09.772450 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:09.772522 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:09.772649 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:09.774430 (+ 1781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:09.775359 (+ 929us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:09.778884 (+ 3525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:09.779302 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:09.779419 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:09.786077 (+ 6658us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:09.786082 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:09.786085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:09.786125 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:09.788551 (+ 2426us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:09.789490 (+ 939us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:09.789494 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:09.789496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:09.789605 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:09.790100 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:09.790103 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:09.790105 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:09.790386 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:09.790483 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:09.791309 (+ 826us) server_negotiation.cc:299] Negotiation successful
0811 22:12:09.791480 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":89,"threads_started":1}
I20260811 22:12:09.795945 11473 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:09.770525 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:41763 (local address 127.28.47.129:41639)
0811 22:12:09.771046 (+ 521us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:09.771103 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:09.772189 (+ 1086us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:09.772865 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:09.772876 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:09.773364 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:09.774172 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:09.774198 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:09.776947 (+ 2749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:09.776951 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:09.778674 (+ 1723us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:09.778694 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:09.779710 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:09.779715 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:09.779985 (+ 270us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:09.780839 (+ 854us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:09.780863 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:09.785500 (+ 4637us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:09.788787 (+ 3287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:09.788797 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:09.788822 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:09.789343 (+ 521us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:09.789732 (+ 389us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:09.789735 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:09.789737 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:09.789975 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:09.790561 (+ 586us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:09.790569 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:09.790970 (+ 401us) client_negotiation.cc:769] Sending connection context
0811 22:12:09.792263 (+ 1293us) client_negotiation.cc:240] Negotiation successful
0811 22:12:09.792712 (+ 449us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":150,"threads_started":1}
I20260811 22:12:09.798192 11466 heartbeater.cc:347] Connected to a master server at 127.28.47.190:41763
I20260811 22:12:09.798542 11466 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:09.799382 11466 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
I20260811 22:12:09.801995 11201 ts_manager.cc:194] Registered new tserver with Master: cb07dda674f44e7698f0cabffacb3b35 (127.28.47.129:36403)
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
I20260811 22:12:09.804194 11201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:41639
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:12:09.830822 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.47.148:44293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:41763
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:12:09 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486329, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260811 22:12:10.011510 11477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:10.012488 11477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:10.012671 11477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:10.020259 11477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:10.020470 11477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:10.020632 11477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:12:10.029940 11477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.28.47.148:44293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:41763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:10.031603 11477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:10.033294 11477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:10.049075 11488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:10.049530 11487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:10.049803 11477 server_base.cc:1034] running on GCE node
W20260811 22:12:10.049089 11490 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:10.050849 11477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:10.051818 11477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:10.053041 11477 hybrid_clock.cc:648] HybridClock initialized: now 1786486330053015 us; error 46 us; skew 500 ppm
Aug 11 22:12:10 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:10.057593 11477 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:12:10.059185 11477 webserver.cc:460] Webserver started at http://127.28.47.130:42255/ using document root <none> and password file <none>
I20260811 22:12:10.060081 11477 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:10.060333 11477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:10.060676 11477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:12:10 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:12:10.079787 11477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "7304252694f14ad5a4732ee0c5824f3d"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-vxj2"
server_key: "3fb2afc0a3d491f833ebdf1a71f0b82a"
server_key_iv: "9f1028fe7107a098703fa520ed393cea"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:10.080714 11477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "7304252694f14ad5a4732ee0c5824f3d"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-vxj2"
server_key: "3fb2afc0a3d491f833ebdf1a71f0b82a"
server_key_iv: "9f1028fe7107a098703fa520ed393cea"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:10.090001 11477 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.007s sys 0.001s
I20260811 22:12:10.119414 11498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:10.121559 11477 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.000s
I20260811 22:12:10.121851 11477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "7304252694f14ad5a4732ee0c5824f3d"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-vxj2"
server_key: "3fb2afc0a3d491f833ebdf1a71f0b82a"
server_key_iv: "9f1028fe7107a098703fa520ed393cea"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:10.122116 11477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:10.159711 11477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:10.164052 11477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:10.164284 11477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:10.165177 11477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:10.166428 11477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:10.166575 11477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:10.166683 11477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:10.166754 11477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:10.179085 11477 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:43721
I20260811 22:12:10.179478 11641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:43721 every 8 connection(s)
I20260811 22:12:10.180554 11477 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:10.182643 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 11477
I20260811 22:12:10.183090 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:12:10.183475 28862 external_mini_cluster.cc:1428] Setting key 159885ea89febbd219c1f5305bda9200
Aug 11 22:12:10 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:10.204350 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:10.183966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:33759 (local address 127.28.47.190:41763)
0811 22:12:10.184158 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:10.184164 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:10.185294 (+ 1130us) server_negotiation.cc:407] Connection header received
0811 22:12:10.189273 (+ 3979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:10.189278 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:10.189352 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:10.189464 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:10.191312 (+ 1848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:10.192113 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:10.194280 (+ 2167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:10.194704 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:10.194834 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:10.198922 (+ 4088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:10.198928 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:10.198932 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:10.198974 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:10.201094 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:10.202056 (+ 962us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:10.202060 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:10.202062 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:10.202163 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:10.202652 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:10.202657 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:10.202659 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:10.202893 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:10.202965 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:10.203943 (+ 978us) server_negotiation.cc:299] Negotiation successful
0811 22:12:10.204139 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:12:10.206753 11646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:10.184355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:41763 (local address 127.28.47.130:33759)
0811 22:12:10.185051 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:10.185121 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:10.186347 (+ 1226us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:10.189770 (+ 3423us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:10.189784 (+ 14us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:10.190233 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:10.191047 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:10.191072 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:10.192298 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:10.192302 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:10.194061 (+ 1759us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:10.194083 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:10.195147 (+ 1064us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:10.195154 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:10.195456 (+ 302us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:10.196341 (+ 885us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:10.196373 (+ 32us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:10.198570 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:10.201344 (+ 2774us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:10.201356 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:10.201385 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:10.201915 (+ 530us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:10.202285 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:10.202288 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:10.202290 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:10.202519 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:10.203041 (+ 522us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:10.203047 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:10.203416 (+ 369us) client_negotiation.cc:769] Sending connection context
0811 22:12:10.204856 (+ 1440us) client_negotiation.cc:240] Negotiation successful
0811 22:12:10.205490 (+ 634us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"thread_start_us":202,"threads_started":1}
I20260811 22:12:10.214481 11642 heartbeater.cc:347] Connected to a master server at 127.28.47.190:41763
I20260811 22:12:10.214815 11642 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:10.215617 11642 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
I20260811 22:12:10.217244 11201 ts_manager.cc:194] Registered new tserver with Master: 7304252694f14ad5a4732ee0c5824f3d (127.28.47.130:43721)
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:12:10.218123 11201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:33759
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:12:10.254343 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.47.148:44293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:41763
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:12:10 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486329, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
W20260811 22:12:10.467103 11649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:10.467681 11649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:10.467814 11649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:10.474929 11649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:10.475128 11649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:10.475284 11649 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:12:10.484414 11649 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.28.47.148:44293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:41763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:10.486425 11649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:10.488274 11649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:10.503611 11656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:10.510044 11655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:10.516870 11659 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:10.519770 11649 server_base.cc:1034] running on GCE node
I20260811 22:12:10.520491 11649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:10.521517 11649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:10.529081 11649 hybrid_clock.cc:648] HybridClock initialized: now 1786486330528976 us; error 115 us; skew 500 ppm
Aug 11 22:12:10 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:10.533740 11649 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:12:10.535344 11649 webserver.cc:460] Webserver started at http://127.28.47.131:43961/ using document root <none> and password file <none>
I20260811 22:12:10.536329 11649 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:10.536507 11649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:10.536832 11649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 11 22:12:10 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:12:10.575446 11649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "713c883b5cf940269f1e1ae888416a75"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-vxj2"
server_key: "7cfcd7934fa0d47f24c04c801b6626e3"
server_key_iv: "80c8350ddb6d07cc40d0119af9622260"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:10.576398 11649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "713c883b5cf940269f1e1ae888416a75"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-vxj2"
server_key: "7cfcd7934fa0d47f24c04c801b6626e3"
server_key_iv: "80c8350ddb6d07cc40d0119af9622260"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:10.583725 11649 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.008s sys 0.000s
I20260811 22:12:10.609673 11666 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:10.611816 11649 fs_manager.cc:725] Time spent opening block manager: real 0.007s user 0.003s sys 0.001s
I20260811 22:12:10.612018 11649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "713c883b5cf940269f1e1ae888416a75"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-vxj2"
server_key: "7cfcd7934fa0d47f24c04c801b6626e3"
server_key_iv: "80c8350ddb6d07cc40d0119af9622260"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:10.612210 11649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:10.649981 11649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:10.654031 11649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:10.654351 11649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:10.655241 11649 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:10.656490 11649 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:10.656615 11649 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:10.656741 11649 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:10.656817 11649 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:10.672818 11649 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:41339
I20260811 22:12:10.674335 11649 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:10.679652 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 11649
I20260811 22:12:10.679800 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:12:10.680122 28862 external_mini_cluster.cc:1428] Setting key 56d6fdb9658afe550eea66aa314c0cc9
I20260811 22:12:10.677686 11804 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:41339 every 8 connection(s)
Aug 11 22:12:10 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:10.702047 11472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:10.678990 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:60733 (local address 127.28.47.190:41763)
0811 22:12:10.679143 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:10.679149 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:10.683636 (+ 4487us) server_negotiation.cc:407] Connection header received
0811 22:12:10.685337 (+ 1701us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:10.685342 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:10.685413 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:10.685476 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:10.687603 (+ 2127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:10.688505 (+ 902us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:10.690590 (+ 2085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:10.690973 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:10.691070 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:10.696879 (+ 5809us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:10.696884 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:10.696887 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:10.696923 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:10.698951 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:10.699835 (+ 884us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:10.699840 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:10.699842 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:10.699951 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:10.700427 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:10.700431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:10.700433 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:10.700725 (+ 292us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:10.700824 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:10.701674 (+ 850us) server_negotiation.cc:299] Negotiation successful
0811 22:12:10.701848 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260811 22:12:10.704272 11807 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:10.679597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:41763 (local address 127.28.47.131:60733)
0811 22:12:10.683426 (+ 3829us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:10.683501 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:10.684931 (+ 1430us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:10.686065 (+ 1134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:10.686077 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:10.686545 (+ 468us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:10.687372 (+ 827us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:10.687397 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:10.688733 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:10.688737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:10.690389 (+ 1652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:10.690405 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:10.691342 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:10.691346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:10.691589 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:10.692398 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:10.692423 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:10.696610 (+ 4187us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:10.699191 (+ 2581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:10.699200 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:10.699225 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:10.699674 (+ 449us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:10.700079 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:10.700083 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:10.700085 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:10.700299 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:10.700907 (+ 608us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:10.700916 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:10.701340 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 22:12:10.702625 (+ 1285us) client_negotiation.cc:240] Negotiation successful
0811 22:12:10.703086 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3531,"thread_start_us":177,"threads_started":1}
I20260811 22:12:10.708151 11805 heartbeater.cc:347] Connected to a master server at 127.28.47.190:41763
I20260811 22:12:10.708482 11805 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:10.709249 11805 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
I20260811 22:12:10.711277 11201 ts_manager.cc:194] Registered new tserver with Master: 713c883b5cf940269f1e1ae888416a75 (127.28.47.131:41339)
I20260811 22:12:10.712116 11201 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:60733
I20260811 22:12:10.722954 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:12:10.723771 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 11297
I20260811 22:12:10.753110 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 11477
I20260811 22:12:10.774662 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 11649
I20260811 22:12:10.800854 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 11163
I20260811 22:12:10.816860 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:41763
--webserver_interface=127.28.47.190
--webserver_port=40011
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:41763
--ranger_config_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.47.148:44293/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.47.148:44293/kms with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:10.942624 11808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:10.942960 11808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:10.943002 11808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:10.947579 11808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:10.947670 11808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:10.947690 11808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:10.947708 11808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:10.947724 11808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:10.954327 11808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.47.148:44293/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:41763
--ranger_config_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:41763
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=40011
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:10.955626 11808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:10.956775 11808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:12:10.966917 11808 server_base.cc:1034] running on GCE node
W20260811 22:12:10.966869 11815 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:10.966801 11816 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:10.966801 11818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:10.967933 11808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:10.969370 11808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:10.970611 11808 hybrid_clock.cc:648] HybridClock initialized: now 1786486330970595 us; error 33 us; skew 500 ppm
Aug 11 22:12:10 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:10.974411 11808 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:12:10.975492 11808 webserver.cc:460] Webserver started at http://127.28.47.190:40011/ using document root <none> and password file <none>
I20260811 22:12:10.976240 11808 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:10.976368 11808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:12:11 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486330, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:12:11.038343 11808 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:12:11.041081 11826 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.042704 11808 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:11.042899 11808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "413c8b4e69b64ef39d7f9a27167aa686"
format_stamp: "Formatted at 2026-08-11 22:12:08 on dist-test-slave-vxj2"
server_key: "7619df2aeafb39e9944645d1f561f547"
server_key_iv: "dcbb0eddc0edb98be876f724d5d12133"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:11.043529 11808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:11.056463 11808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:11.059844 11808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:11.060133 11808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:11.069263 11808 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:41763
I20260811 22:12:11.069298 11883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:41763 every 8 connection(s)
I20260811 22:12:11.070607 11808 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:11.076951 11885 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:12:11.076918 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 11808
I20260811 22:12:11.077553 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.47.148:44293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:36403
--local_ip_for_outbound_sockets=127.28.47.129
--tserver_master_addrs=127.28.47.190:41763
--webserver_port=34413
--webserver_interface=127.28.47.129
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.47.148:44293/kms with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:12:11.079744 11885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686: Bootstrap starting.
I20260811 22:12:11.103344 11885 log.cc:824] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:11.111534 11885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:12:11.112043 11885 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686: Bootstrap complete.
I20260811 22:12:11.116528 11885 raft_consensus.cc:348] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413c8b4e69b64ef39d7f9a27167aa686" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 41763 } }
I20260811 22:12:11.117043 11885 raft_consensus.cc:729] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413c8b4e69b64ef39d7f9a27167aa686, State: Initialized, Role: FOLLOWER
I20260811 22:12:11.117657 11885 consensus_queue.cc:260] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413c8b4e69b64ef39d7f9a27167aa686" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 41763 } }
I20260811 22:12:11.117828 11885 raft_consensus.cc:388] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:11.117928 11885 raft_consensus.cc:482] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:11.118059 11885 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:12:11.120196 11885 raft_consensus.cc:504] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413c8b4e69b64ef39d7f9a27167aa686" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 41763 } }
I20260811 22:12:11.120620 11885 leader_election.cc:302] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 413c8b4e69b64ef39d7f9a27167aa686; no voters:
I20260811 22:12:11.121011 11885 leader_election.cc:288] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:12:11.121114 11890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:12:11.121330 11890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [term 2 LEADER]: Becoming Leader. State: Replica: 413c8b4e69b64ef39d7f9a27167aa686, State: Running, Role: LEADER
I20260811 22:12:11.121719 11890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413c8b4e69b64ef39d7f9a27167aa686" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 41763 } }
I20260811 22:12:11.122453 11885 sys_catalog.cc:564] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:11.123267 11891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "413c8b4e69b64ef39d7f9a27167aa686" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413c8b4e69b64ef39d7f9a27167aa686" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 41763 } } }
I20260811 22:12:11.123456 11891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:11.123929 11898 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:11.124013 11892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 413c8b4e69b64ef39d7f9a27167aa686. Latest consensus state: current_term: 2 leader_uuid: "413c8b4e69b64ef39d7f9a27167aa686" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413c8b4e69b64ef39d7f9a27167aa686" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 41763 } } }
I20260811 22:12:11.124203 11892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:11.127841 11898 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:11.128459 11885 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260811 22:12:11.128746 11898 catalog_manager.cc:1259] Loaded cluster ID: 2180ad5b1f294969b7c561680166539e
I20260811 22:12:11.128790 11898 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:11.135469 11898 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:11.136760 11898 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686: Loaded TSK: 0
I20260811 22:12:11.138618 11898 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:12:11.222260 11887 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:11.222712 11887 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:11.222860 11887 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:11.230037 11887 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:11.230134 11887 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:11.230237 11887 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:12:11.239321 11887 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.47.148:44293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:36403
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=34413
--tserver_master_addrs=127.28.47.190:41763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11887
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:11.240994 11887 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:11.242522 11887 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:11.256439 11930 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:11.256610 11928 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:11.256407 11927 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:11.257232 11887 server_base.cc:1034] running on GCE node
I20260811 22:12:11.257819 11887 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:11.258792 11887 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:11.260010 11887 hybrid_clock.cc:648] HybridClock initialized: now 1786486331259976 us; error 53 us; skew 500 ppm
Aug 11 22:12:11 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486331, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:11.263942 11887 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:12:11.265249 11887 webserver.cc:460] Webserver started at http://127.28.47.129:34413/ using document root <none> and password file <none>
I20260811 22:12:11.266077 11887 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:11.266209 11887 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:12:11 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486331, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:12:11.295312 11887 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:12:11.300686 11942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.302783 11887 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.003s sys 0.000s
I20260811 22:12:11.302959 11887 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "cb07dda674f44e7698f0cabffacb3b35"
format_stamp: "Formatted at 2026-08-11 22:12:09 on dist-test-slave-vxj2"
server_key: "86ce2eeeb5352bd9cfd5a17ca687ddef"
server_key_iv: "15a06df09b61979de9d821d938ae060a"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:11.303788 11887 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:11.340323 11887 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:11.344446 11887 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:11.344735 11887 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:11.345539 11887 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:11.346666 11887 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:11.346778 11887 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.346901 11887 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:11.346979 11887 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.361083 11887 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:36403
I20260811 22:12:11.361243 12075 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:36403 every 8 connection(s)
I20260811 22:12:11.362514 11887 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:11.370677 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 11887
I20260811 22:12:11.371187 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.47.148:44293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:43721
--local_ip_for_outbound_sockets=127.28.47.130
--tserver_master_addrs=127.28.47.190:41763
--webserver_port=42255
--webserver_interface=127.28.47.130
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.47.148:44293/kms with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:12:11 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486331, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:11.398994 12078 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.366807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:41763 (local address 127.28.47.129:42387)
0811 22:12:11.367394 (+ 587us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:11.367453 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:11.368512 (+ 1059us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:11.370158 (+ 1646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:11.370167 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:11.370566 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:11.371287 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:11.371314 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.375974 (+ 4660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.375979 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:11.377680 (+ 1701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:11.377700 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.378976 (+ 1276us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.378981 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:11.379206 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:11.380164 (+ 958us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:11.380192 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:11.389730 (+ 9538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:11.393804 (+ 4074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:11.393815 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:11.393834 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:11.394315 (+ 481us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:11.394878 (+ 563us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:11.394881 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:11.394896 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:11.395112 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:11.396071 (+ 959us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:11.396082 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:11.396501 (+ 419us) client_negotiation.cc:769] Sending connection context
0811 22:12:11.396903 (+ 402us) client_negotiation.cc:240] Negotiation successful
0811 22:12:11.397417 (+ 514us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":178,"threads_started":1}
I20260811 22:12:11.402454 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.368370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:42387 (local address 127.28.47.190:41763)
0811 22:12:11.369024 (+ 654us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.369043 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.369081 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:12:11.369408 (+ 327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.369419 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.369735 (+ 316us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.370076 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.373932 (+ 3856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.375698 (+ 1766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.377908 (+ 2210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.378720 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.379698 (+ 978us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:11.390159 (+ 10461us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.390168 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.390193 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:11.390245 (+ 52us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.393558 (+ 3313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:11.394525 (+ 967us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:11.394533 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:11.394539 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:11.394735 (+ 196us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:11.395246 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:11.395249 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:11.395251 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:11.395854 (+ 603us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.396019 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:11.399924 (+ 3905us) server_negotiation.cc:299] Negotiation successful
0811 22:12:11.401140 (+ 1216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":494,"thread_start_us":121,"threads_started":1}
I20260811 22:12:11.405876 12076 heartbeater.cc:347] Connected to a master server at 127.28.47.190:41763
I20260811 22:12:11.406200 12076 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:11.406936 12076 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
W20260811 22:12:11.410145 12076 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:41763 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:12:11.410341 12076 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:11.410589 12076 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
I20260811 22:12:11.411944 12076 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:11.412199 12076 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
W20260811 22:12:11.413278 12076 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:12:11.556291 12080 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:11.556728 12080 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:11.556865 12080 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:11.561568 12080 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:11.561648 12080 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:11.561764 12080 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:12:11.567988 12080 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.47.148:44293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:43721
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=42255
--tserver_master_addrs=127.28.47.190:41763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12080
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:11.569353 12080 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:11.570606 12080 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:11.584470 12085 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:11.584594 12080 server_base.cc:1034] running on GCE node
W20260811 22:12:11.584482 12088 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:11.584785 12086 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:11.585753 12080 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:11.586771 12080 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:11.587935 12080 hybrid_clock.cc:648] HybridClock initialized: now 1786486331587910 us; error 42 us; skew 500 ppm
Aug 11 22:12:11 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486331, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:11.595461 12080 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:12:11.596709 12080 webserver.cc:460] Webserver started at http://127.28.47.130:42255/ using document root <none> and password file <none>
I20260811 22:12:11.597450 12080 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:11.597581 12080 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:12:11 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486331, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:12:11.622476 12080 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:12:11.627990 12096 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.630113 12080 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260811 22:12:11.630306 12080 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "7304252694f14ad5a4732ee0c5824f3d"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-vxj2"
server_key: "3fb2afc0a3d491f833ebdf1a71f0b82a"
server_key_iv: "9f1028fe7107a098703fa520ed393cea"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:11.631165 12080 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:11.671811 12080 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:11.676115 12080 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:11.676437 12080 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:11.677378 12080 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:11.678817 12080 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:11.678895 12080 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.678966 12080 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:11.679001 12080 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.691812 12080 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:43721
I20260811 22:12:11.693485 12080 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:11.694626 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 12080
I20260811 22:12:11.694880 12228 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:43721 every 8 connection(s)
I20260811 22:12:11.695127 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.28.47.148:44293/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:41339
--local_ip_for_outbound_sockets=127.28.47.131
--tserver_master_addrs=127.28.47.190:41763
--webserver_port=43961
--webserver_interface=127.28.47.131
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39721
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.28.47.148:44293/kms with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:12:11 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486331, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:11.717360 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.699506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:38765 (local address 127.28.47.190:41763)
0811 22:12:11.700679 (+ 1173us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.700687 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.701224 (+ 537us) server_negotiation.cc:407] Connection header received
0811 22:12:11.702658 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.702663 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.702733 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.702794 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.704639 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.705498 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.707639 (+ 2141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.708144 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.708263 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:11.712276 (+ 4013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.712281 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.712284 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:11.712331 (+ 47us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.714392 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:11.715186 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:11.715190 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:11.715192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:11.715310 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:11.715792 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:11.715799 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:11.715801 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:11.716101 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.716225 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:11.716955 (+ 730us) server_negotiation.cc:299] Negotiation successful
0811 22:12:11.717125 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1031}
I20260811 22:12:11.718649 12235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.699471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:41763 (local address 127.28.47.130:38765)
0811 22:12:11.701017 (+ 1546us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:11.701080 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:11.702354 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:11.703269 (+ 915us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:11.703279 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:11.703680 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:11.704434 (+ 754us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:11.704456 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.705706 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.705712 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:11.707448 (+ 1736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:11.707466 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.708551 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.708558 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:11.708845 (+ 287us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:11.709712 (+ 867us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:11.709741 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:11.711977 (+ 2236us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:11.714580 (+ 2603us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:11.714589 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:11.714612 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:11.715041 (+ 429us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:11.715442 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:11.715445 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:11.715447 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:11.715664 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:11.716228 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:11.716236 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:11.716635 (+ 399us) client_negotiation.cc:769] Sending connection context
0811 22:12:11.716869 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 22:12:11.717293 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1290,"thread_start_us":180,"threads_started":1}
I20260811 22:12:11.720486 12231 heartbeater.cc:347] Connected to a master server at 127.28.47.190:41763
I20260811 22:12:11.720803 12231 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:11.721645 12231 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
W20260811 22:12:11.723794 12231 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:41763 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:12:11.723971 12231 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:11.724212 12231 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
I20260811 22:12:11.725800 12231 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:11.726143 12231 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
W20260811 22:12:11.727257 12231 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:12:11.860649 12233 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:11.861045 12233 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:11.861219 12233 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:11.866336 12233 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:11.866443 12233 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:11.866580 12233 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:12:11.873860 12233 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39721
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.28.47.148:44293/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:41339
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=43961
--tserver_master_addrs=127.28.47.190:41763
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12233
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:11.875672 12233 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:11.877374 12233 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:11.889094 12241 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:11.889271 12240 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:11.889094 12244 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:11.890252 12233 server_base.cc:1034] running on GCE node
I20260811 22:12:11.890800 12233 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:11.891721 12233 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:11.893141 12233 hybrid_clock.cc:648] HybridClock initialized: now 1786486331893100 us; error 60 us; skew 500 ppm
Aug 11 22:12:11 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486331, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:11.896919 12233 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:12:11.898164 12233 webserver.cc:460] Webserver started at http://127.28.47.131:43961/ using document root <none> and password file <none>
I20260811 22:12:11.898873 12233 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:11.899016 12233 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 11 22:12:11 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486331, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:12:11.926733 12233 fs_manager.cc:709] Time spent opening directory manager: real 0.005s user 0.002s sys 0.001s
I20260811 22:12:11.930600 12252 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.932262 12233 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.001s
I20260811 22:12:11.932468 12233 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "713c883b5cf940269f1e1ae888416a75"
format_stamp: "Formatted at 2026-08-11 22:12:10 on dist-test-slave-vxj2"
server_key: "7cfcd7934fa0d47f24c04c801b6626e3"
server_key_iv: "80c8350ddb6d07cc40d0119af9622260"
server_key_version: "kuduclusterkey@0"
I20260811 22:12:11.933140 12233 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:11.958684 12233 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:11.962977 12233 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:11.963287 12233 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:11.964174 12233 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:11.965415 12233 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:11.965525 12233 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.965631 12233 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:11.965696 12233 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:11.980065 12233 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:41339
I20260811 22:12:11.981792 12233 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:11.982872 12390 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:41339 every 8 connection(s)
I20260811 22:12:11.989890 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 12233
Aug 11 22:12:11 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486331, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:12.001032 12079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.984416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:59701 (local address 127.28.47.190:41763)
0811 22:12:11.984608 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.984613 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.985670 (+ 1057us) server_negotiation.cc:407] Connection header received
0811 22:12:11.986762 (+ 1092us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.986768 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.986843 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.986967 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.988426 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.989332 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.991866 (+ 2534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.992237 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.992394 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:11.996039 (+ 3645us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.996043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.996047 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:11.996080 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.997848 (+ 1768us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:11.998927 (+ 1079us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:11.998932 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:11.998934 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:11.999038 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:11.999488 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:11.999494 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:11.999496 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:11.999751 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:11.999866 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:12.000700 (+ 834us) server_negotiation.cc:299] Negotiation successful
0811 22:12:12.000854 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:12:12.001302 12396 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.990998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46644 (local address 127.28.47.190:41763)
0811 22:12:11.991306 (+ 308us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:11.991310 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:11.991615 (+ 305us) server_negotiation.cc:407] Connection header received
0811 22:12:11.991861 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:11.991864 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:11.991925 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:11.991989 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:11.993050 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.993893 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.994866 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.995159 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.995283 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:11.996767 (+ 1484us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:11.996771 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:11.996774 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:11.996818 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:11.999339 (+ 2521us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:11.999906 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:11.999908 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:11.999910 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:11.999995 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:12.000301 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:12.000304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:12.000306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:12.000475 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:12.000699 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:12.000887 (+ 188us) server_negotiation.cc:299] Negotiation successful
0811 22:12:12.001009 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":231,"thread_start_us":118,"threads_started":1}
I20260811 22:12:12.003073 12393 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:11.984892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:41763 (local address 127.28.47.131:59701)
0811 22:12:11.985466 (+ 574us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:11.985528 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:11.986490 (+ 962us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:11.987222 (+ 732us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:11.987230 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:11.987566 (+ 336us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:11.988174 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:11.988192 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.989625 (+ 1433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.989636 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:11.991639 (+ 2003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:11.991663 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:11.992476 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:11.992484 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:11.992746 (+ 262us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:11.993522 (+ 776us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:11.993542 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:11.995738 (+ 2196us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:11.997988 (+ 2250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:11.997999 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:11.998021 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:11.998787 (+ 766us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:11.999174 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:11.999179 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:11.999182 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:11.999404 (+ 222us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:11.999903 (+ 499us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:11.999910 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:12.000334 (+ 424us) client_negotiation.cc:769] Sending connection context
0811 22:12:12.001423 (+ 1089us) client_negotiation.cc:240] Negotiation successful
0811 22:12:12.001858 (+ 435us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":210,"threads_started":1}
I20260811 22:12:12.004653 12391 heartbeater.cc:347] Connected to a master server at 127.28.47.190:41763
I20260811 22:12:12.005035 12391 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:12.005818 12391 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
W20260811 22:12:12.008054 12391 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:41763 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260811 22:12:12.008292 12391 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:12.008622 12391 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
I20260811 22:12:12.010455 12391 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:12.010758 12391 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
W20260811 22:12:12.012050 12391 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 11 22:12:12 dist-test-slave-vxj2 krb5kdc[10608](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786486332, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:12.413621 12076 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:12.413985 12076 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
I20260811 22:12:12.415515 11845 ts_manager.cc:194] Registered new tserver with Master: cb07dda674f44e7698f0cabffacb3b35 (127.28.47.129:36403)
I20260811 22:12:12.417385 11845 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:42387
I20260811 22:12:12.727641 12231 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:12.728163 12231 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
I20260811 22:12:12.729482 11844 ts_manager.cc:194] Registered new tserver with Master: 7304252694f14ad5a4732ee0c5824f3d (127.28.47.130:43721)
I20260811 22:12:12.730432 11844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:38765
Aug 11 22:12:12 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486332, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:12:13.012395 12391 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:13.012746 12391 heartbeater.cc:511] Master 127.28.47.190:41763 requested a full tablet report, sending...
I20260811 22:12:13.013760 11844 ts_manager.cc:194] Registered new tserver with Master: 713c883b5cf940269f1e1ae888416a75 (127.28.47.131:41339)
I20260811 22:12:13.014549 11844 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:59701
I20260811 22:12:13.021471 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:12:13.035084 12418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.023603 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46658 (local address 127.28.47.190:41763)
0811 22:12:13.024076 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:13.024081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:13.024387 (+ 306us) server_negotiation.cc:407] Connection header received
0811 22:12:13.024578 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:13.024582 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:13.024662 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:13.024721 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:13.025705 (+ 984us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.026519 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.027743 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.028163 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.028264 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.030435 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:13.030441 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:13.030446 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:13.030491 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:13.032517 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:13.033079 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:13.033083 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:13.033085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:13.033215 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:13.033613 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:13.033617 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:13.033619 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:13.033869 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:13.033972 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:13.034744 (+ 772us) server_negotiation.cc:299] Negotiation successful
0811 22:12:13.034896 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":109,"threads_started":1}
I20260811 22:12:13.040482 11844 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46658:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:12:13.045553 11844 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:12:13.080421 12428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.066850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:43721 (local address 127.0.0.1:53618)
0811 22:12:13.068885 (+ 2035us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:13.068899 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:13.068991 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:13.069885 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:13.069891 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:13.069915 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:13.070058 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.070064 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.071863 (+ 1799us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.071867 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.073378 (+ 1511us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.073391 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.079814 (+ 6423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.079817 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.079967 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.080008 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 22:12:13.080120 (+ 112us) client_negotiation.cc:240] Negotiation successful
0811 22:12:13.080262 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1960,"thread_start_us":97,"threads_started":1}
I20260811 22:12:13.085462 12427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.066924 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:41339 (local address 127.0.0.1:47722)
0811 22:12:13.073651 (+ 6727us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:13.073666 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:13.073753 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:13.075156 (+ 1403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:13.075160 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:13.075179 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:13.075293 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.075299 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.076807 (+ 1508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.076810 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.078073 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.078085 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.084881 (+ 6796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.084886 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.085017 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.085048 (+ 31us) client_negotiation.cc:769] Sending connection context
0811 22:12:13.085133 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:12:13.085275 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6649,"thread_start_us":102,"threads_started":1}
I20260811 22:12:13.086177 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.067031 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53618 (local address 127.28.47.130:43721)
0811 22:12:13.068744 (+ 1713us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:13.068752 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:13.069080 (+ 328us) server_negotiation.cc:407] Connection header received
0811 22:12:13.069265 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:13.069272 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:13.069513 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:13.069639 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:13.070431 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.071697 (+ 1266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.078246 (+ 6549us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.079182 (+ 936us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.079300 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.079468 (+ 168us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:13.085867 (+ 6399us) server_negotiation.cc:299] Negotiation successful
0811 22:12:13.086022 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1596,"thread_start_us":103,"threads_started":1}
I20260811 22:12:13.090377 12147 tablet_service.cc:1467] Processing CreateTablet for tablet 2aef699d42de4435a2c4bf70ffead64b (DEFAULT_TABLE table=test-table [id=c3875afdb050475f97963029079787b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:13.092340 12147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aef699d42de4435a2c4bf70ffead64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:13.102387 12430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.067330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47722 (local address 127.28.47.131:41339)
0811 22:12:13.070359 (+ 3029us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:13.070367 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:13.073875 (+ 3508us) server_negotiation.cc:407] Connection header received
0811 22:12:13.073978 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:13.073985 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:13.074245 (+ 260us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:13.074391 (+ 146us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:13.075421 (+ 1030us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.076645 (+ 1224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.083439 (+ 6794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.084476 (+ 1037us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.084610 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.084791 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:13.101998 (+ 17207us) server_negotiation.cc:299] Negotiation successful
0811 22:12:13.102170 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2909,"thread_start_us":185,"threads_started":1}
I20260811 22:12:13.106650 12305 tablet_service.cc:1467] Processing CreateTablet for tablet 2aef699d42de4435a2c4bf70ffead64b (DEFAULT_TABLE table=test-table [id=c3875afdb050475f97963029079787b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:13.108234 12305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aef699d42de4435a2c4bf70ffead64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:13.122226 12425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.065092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:36403 (local address 127.0.0.1:37662)
0811 22:12:13.068134 (+ 3042us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:13.068198 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:13.068336 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:13.079572 (+ 11236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:13.079575 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:13.079604 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:13.079709 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.079715 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.098815 (+ 19100us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.098820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.100389 (+ 1569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.100413 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.121488 (+ 21075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.121494 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.121728 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.121770 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 22:12:13.121847 (+ 77us) client_negotiation.cc:240] Negotiation successful
0811 22:12:13.121956 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2838,"thread_start_us":174,"threads_started":1}
I20260811 22:12:13.124976 12426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.065264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37662 (local address 127.28.47.129:36403)
0811 22:12:13.074716 (+ 9452us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:13.074724 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:13.074748 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:12:13.074822 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:13.074827 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:13.075016 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:13.080893 (+ 5877us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:13.081066 (+ 173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.083222 (+ 2156us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.100663 (+ 17441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.101771 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.123030 (+ 21259us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.123687 (+ 657us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:13.124514 (+ 827us) server_negotiation.cc:299] Negotiation successful
0811 22:12:13.124747 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":9329,"thread_start_us":135,"threads_started":1}
I20260811 22:12:13.129739 12432 tablet_bootstrap.cc:492] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d: Bootstrap starting.
I20260811 22:12:13.131436 12431 tablet_bootstrap.cc:492] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75: Bootstrap starting.
I20260811 22:12:13.132731 11988 tablet_service.cc:1467] Processing CreateTablet for tablet 2aef699d42de4435a2c4bf70ffead64b (DEFAULT_TABLE table=test-table [id=c3875afdb050475f97963029079787b4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:13.134327 11988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2aef699d42de4435a2c4bf70ffead64b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:13.134387 12432 tablet_bootstrap.cc:654] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:13.135926 12432 log.cc:824] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:13.139979 12431 tablet_bootstrap.cc:654] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:13.141206 12431 log.cc:824] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:13.144310 12432 tablet_bootstrap.cc:492] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d: No bootstrap required, opened a new log
I20260811 22:12:13.144695 12432 ts_tablet_manager.cc:1397] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d: Time spent bootstrapping tablet: real 0.015s user 0.007s sys 0.000s
I20260811 22:12:13.150571 12432 raft_consensus.cc:348] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb07dda674f44e7698f0cabffacb3b35" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 36403 } } peers { permanent_uuid: "7304252694f14ad5a4732ee0c5824f3d" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 43721 } } peers { permanent_uuid: "713c883b5cf940269f1e1ae888416a75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 41339 } }
I20260811 22:12:13.151153 12432 raft_consensus.cc:374] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:13.151264 12432 raft_consensus.cc:729] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7304252694f14ad5a4732ee0c5824f3d, State: Initialized, Role: FOLLOWER
I20260811 22:12:13.151957 12432 consensus_queue.cc:260] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb07dda674f44e7698f0cabffacb3b35" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 36403 } } peers { permanent_uuid: "7304252694f14ad5a4732ee0c5824f3d" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 43721 } } peers { permanent_uuid: "713c883b5cf940269f1e1ae888416a75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 41339 } }
I20260811 22:12:13.153482 12432 ts_tablet_manager.cc:1428] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d: Time spent starting tablet: real 0.009s user 0.003s sys 0.005s
I20260811 22:12:13.153949 12231 heartbeater.cc:503] Master 127.28.47.190:41763 was elected leader, sending a full tablet report...
I20260811 22:12:13.157646 12435 tablet_bootstrap.cc:492] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35: Bootstrap starting.
I20260811 22:12:13.162669 12431 tablet_bootstrap.cc:492] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75: No bootstrap required, opened a new log
I20260811 22:12:13.163050 12431 ts_tablet_manager.cc:1397] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75: Time spent bootstrapping tablet: real 0.032s user 0.008s sys 0.000s
I20260811 22:12:13.167783 12431 raft_consensus.cc:348] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb07dda674f44e7698f0cabffacb3b35" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 36403 } } peers { permanent_uuid: "7304252694f14ad5a4732ee0c5824f3d" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 43721 } } peers { permanent_uuid: "713c883b5cf940269f1e1ae888416a75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 41339 } }
I20260811 22:12:13.168169 12431 raft_consensus.cc:374] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:13.168231 12431 raft_consensus.cc:729] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 713c883b5cf940269f1e1ae888416a75, State: Initialized, Role: FOLLOWER
I20260811 22:12:13.169061 12431 consensus_queue.cc:260] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb07dda674f44e7698f0cabffacb3b35" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 36403 } } peers { permanent_uuid: "7304252694f14ad5a4732ee0c5824f3d" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 43721 } } peers { permanent_uuid: "713c883b5cf940269f1e1ae888416a75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 41339 } }
I20260811 22:12:13.171113 12435 tablet_bootstrap.cc:654] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:13.172487 12431 ts_tablet_manager.cc:1428] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260811 22:12:13.173007 12391 heartbeater.cc:503] Master 127.28.47.190:41763 was elected leader, sending a full tablet report...
I20260811 22:12:13.173552 12435 log.cc:824] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:13.176923 12435 tablet_bootstrap.cc:492] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35: No bootstrap required, opened a new log
I20260811 22:12:13.177212 12435 ts_tablet_manager.cc:1397] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35: Time spent bootstrapping tablet: real 0.020s user 0.007s sys 0.000s
I20260811 22:12:13.181977 12435 raft_consensus.cc:348] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb07dda674f44e7698f0cabffacb3b35" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 36403 } } peers { permanent_uuid: "7304252694f14ad5a4732ee0c5824f3d" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 43721 } } peers { permanent_uuid: "713c883b5cf940269f1e1ae888416a75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 41339 } }
I20260811 22:12:13.182389 12435 raft_consensus.cc:374] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:13.182441 12435 raft_consensus.cc:729] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb07dda674f44e7698f0cabffacb3b35, State: Initialized, Role: FOLLOWER
I20260811 22:12:13.183079 12435 consensus_queue.cc:260] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb07dda674f44e7698f0cabffacb3b35" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 36403 } } peers { permanent_uuid: "7304252694f14ad5a4732ee0c5824f3d" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 43721 } } peers { permanent_uuid: "713c883b5cf940269f1e1ae888416a75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 41339 } }
I20260811 22:12:13.184016 12076 heartbeater.cc:503] Master 127.28.47.190:41763 was elected leader, sending a full tablet report...
I20260811 22:12:13.184407 12435 ts_tablet_manager.cc:1428] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260811 22:12:13.197180 12232 tablet.cc:2369] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:12:13.230070 12437 raft_consensus.cc:482] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:13.230340 12437 raft_consensus.cc:504] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb07dda674f44e7698f0cabffacb3b35" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 36403 } } peers { permanent_uuid: "7304252694f14ad5a4732ee0c5824f3d" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 43721 } } peers { permanent_uuid: "713c883b5cf940269f1e1ae888416a75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 41339 } }
I20260811 22:12:13.231855 12437 leader_election.cc:288] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cb07dda674f44e7698f0cabffacb3b35 (127.28.47.129:36403), 7304252694f14ad5a4732ee0c5824f3d (127.28.47.130:43721)
I20260811 22:12:13.232491 12439 raft_consensus.cc:482] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:13.232684 12439 raft_consensus.cc:504] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb07dda674f44e7698f0cabffacb3b35" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 36403 } } peers { permanent_uuid: "7304252694f14ad5a4732ee0c5824f3d" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 43721 } } peers { permanent_uuid: "713c883b5cf940269f1e1ae888416a75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 41339 } }
W20260811 22:12:13.234305 12392 tablet.cc:2369] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:12:13.234526 12439 leader_election.cc:288] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7304252694f14ad5a4732ee0c5824f3d (127.28.47.130:43721), 713c883b5cf940269f1e1ae888416a75 (127.28.47.131:41339)
I20260811 22:12:13.240149 12426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.233909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:45251 (local address 127.28.47.129:36403)
0811 22:12:13.234443 (+ 534us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:13.234447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:13.234465 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:12:13.235560 (+ 1095us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:13.235564 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:13.235616 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:13.235667 (+ 51us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:13.236146 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.237068 (+ 922us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.238873 (+ 1805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.239405 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.239533 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.239583 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:13.239923 (+ 340us) server_negotiation.cc:299] Negotiation successful
0811 22:12:13.240005 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":462}
I20260811 22:12:13.240664 12442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.232277 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:36403 (local address 127.28.47.131:45251)
0811 22:12:13.233958 (+ 1681us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:13.233976 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:13.234107 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:13.235837 (+ 1730us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:13.235840 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:13.235862 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:13.236001 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.236008 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.237234 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.237237 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.238721 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.238733 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.239651 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.239655 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.239758 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.239785 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:12:13.240386 (+ 601us) client_negotiation.cc:240] Negotiation successful
0811 22:12:13.240481 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1597,"mutex_wait_us":1,"thread_start_us":153,"threads_started":1}
I20260811 22:12:13.241402 12002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aef699d42de4435a2c4bf70ffead64b" candidate_uuid: "713c883b5cf940269f1e1ae888416a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb07dda674f44e7698f0cabffacb3b35" is_pre_election: true
I20260811 22:12:13.241715 12002 raft_consensus.cc:2454] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 713c883b5cf940269f1e1ae888416a75 in term 0.
I20260811 22:12:13.242417 12254 leader_election.cc:302] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 713c883b5cf940269f1e1ae888416a75, cb07dda674f44e7698f0cabffacb3b35; no voters:
I20260811 22:12:13.242764 12437 raft_consensus.cc:2781] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:13.242837 12437 raft_consensus.cc:482] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:13.242895 12437 raft_consensus.cc:3037] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:13.244313 12437 raft_consensus.cc:504] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb07dda674f44e7698f0cabffacb3b35" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 36403 } } peers { permanent_uuid: "7304252694f14ad5a4732ee0c5824f3d" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 43721 } } peers { permanent_uuid: "713c883b5cf940269f1e1ae888416a75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 41339 } }
I20260811 22:12:13.244638 12429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.233382 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:39709 (local address 127.28.47.130:43721)
0811 22:12:13.233534 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:13.233541 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:13.235259 (+ 1718us) server_negotiation.cc:407] Connection header received
0811 22:12:13.235478 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:13.235482 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:13.235535 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:13.235621 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:13.236059 (+ 438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.236877 (+ 818us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.238651 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.239277 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.239410 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.239508 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:13.244399 (+ 4891us) server_negotiation.cc:299] Negotiation successful
0811 22:12:13.244492 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":18,"server-negotiator.queue_time_us":62}
I20260811 22:12:13.244751 12437 leader_election.cc:288] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [CANDIDATE]: Term 1 election: Requested vote from peers cb07dda674f44e7698f0cabffacb3b35 (127.28.47.129:36403), 7304252694f14ad5a4732ee0c5824f3d (127.28.47.130:43721)
I20260811 22:12:13.245179 12444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.232761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:43721 (local address 127.28.47.131:39709)
0811 22:12:13.235150 (+ 2389us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:13.235167 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:13.235343 (+ 176us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:13.235746 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:13.235749 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:13.235768 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:13.235926 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.235933 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.237047 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.237051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.238472 (+ 1421us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.238486 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.241915 (+ 3429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.241919 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.242031 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.242061 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:12:13.244869 (+ 2808us) client_negotiation.cc:240] Negotiation successful
0811 22:12:13.244977 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2334,"thread_start_us":201,"threads_started":1}
I20260811 22:12:13.245734 12002 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aef699d42de4435a2c4bf70ffead64b" candidate_uuid: "713c883b5cf940269f1e1ae888416a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cb07dda674f44e7698f0cabffacb3b35"
I20260811 22:12:13.245898 12002 raft_consensus.cc:3037] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:13.246188 12176 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aef699d42de4435a2c4bf70ffead64b" candidate_uuid: "713c883b5cf940269f1e1ae888416a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7304252694f14ad5a4732ee0c5824f3d" is_pre_election: true
I20260811 22:12:13.246583 12176 raft_consensus.cc:2454] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 713c883b5cf940269f1e1ae888416a75 in term 0.
I20260811 22:12:13.246956 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aef699d42de4435a2c4bf70ffead64b" candidate_uuid: "713c883b5cf940269f1e1ae888416a75" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7304252694f14ad5a4732ee0c5824f3d"
I20260811 22:12:13.247094 12175 raft_consensus.cc:3037] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:13.247359 12002 raft_consensus.cc:2454] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 713c883b5cf940269f1e1ae888416a75 in term 1.
I20260811 22:12:13.247960 12254 leader_election.cc:302] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 713c883b5cf940269f1e1ae888416a75, cb07dda674f44e7698f0cabffacb3b35; no voters:
I20260811 22:12:13.248215 12437 raft_consensus.cc:2781] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:13.248534 12437 raft_consensus.cc:686] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [term 1 LEADER]: Becoming Leader. State: Replica: 713c883b5cf940269f1e1ae888416a75, State: Running, Role: LEADER
I20260811 22:12:13.248718 12175 raft_consensus.cc:2454] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 713c883b5cf940269f1e1ae888416a75 in term 1.
Aug 11 22:12:13 dist-test-slave-vxj2 krb5kdc[10608](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486332, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for HTTP/127.28.47.148@KRBTEST.COM
I20260811 22:12:13.248998 12437 consensus_queue.cc:237] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb07dda674f44e7698f0cabffacb3b35" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 36403 } } peers { permanent_uuid: "7304252694f14ad5a4732ee0c5824f3d" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 43721 } } peers { permanent_uuid: "713c883b5cf940269f1e1ae888416a75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 41339 } }
I20260811 22:12:13.250371 12446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.234132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:43721 (local address 127.28.47.129:38345)
0811 22:12:13.239038 (+ 4906us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:13.239055 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:13.239165 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:13.239650 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:13.239653 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:13.239674 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:13.239857 (+ 183us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.239865 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.241444 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.241448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.243253 (+ 1805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.243265 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.249756 (+ 6491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.249761 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.249930 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.249972 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 22:12:13.250083 (+ 111us) client_negotiation.cc:240] Negotiation successful
0811 22:12:13.250183 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4813,"thread_start_us":110,"threads_started":1}
I20260811 22:12:13.250404 12448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.234646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:38345 (local address 127.28.47.130:43721)
0811 22:12:13.235037 (+ 391us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:13.235045 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:13.239256 (+ 4211us) server_negotiation.cc:407] Connection header received
0811 22:12:13.239310 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:13.239315 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:13.239373 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:13.239443 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:13.240008 (+ 565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.241282 (+ 1274us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.243418 (+ 2136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.244099 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.244242 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.244310 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:13.250133 (+ 5823us) server_negotiation.cc:299] Negotiation successful
0811 22:12:13.250248 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":312,"thread_start_us":110,"threads_started":1}
I20260811 22:12:13.251192 12175 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aef699d42de4435a2c4bf70ffead64b" candidate_uuid: "cb07dda674f44e7698f0cabffacb3b35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7304252694f14ad5a4732ee0c5824f3d" is_pre_election: true
I20260811 22:12:13.251405 12175 raft_consensus.cc:2379] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate cb07dda674f44e7698f0cabffacb3b35 in current term 1: Already voted for candidate 713c883b5cf940269f1e1ae888416a75 in this term.
I20260811 22:12:13.253551 12430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.234924 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:50795 (local address 127.28.47.131:41339)
0811 22:12:13.235041 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:13.235045 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:13.241484 (+ 6439us) server_negotiation.cc:407] Connection header received
0811 22:12:13.241679 (+ 195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:13.241682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:13.241732 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:13.241781 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:13.242194 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.242864 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.244385 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.244919 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.245020 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.245101 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:13.253295 (+ 8194us) server_negotiation.cc:299] Negotiation successful
0811 22:12:13.253427 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260811 22:12:13.254025 12447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.234179 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:41339 (local address 127.28.47.129:50795)
0811 22:12:13.241380 (+ 7201us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:13.241398 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:13.241557 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:13.241921 (+ 364us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:13.241924 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:13.241940 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:13.242068 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.242074 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.243024 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.243027 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.244214 (+ 1187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:13.244229 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.245231 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.245234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:13.245374 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.245416 (+ 42us) client_negotiation.cc:769] Sending connection context
0811 22:12:13.253769 (+ 8353us) client_negotiation.cc:240] Negotiation successful
0811 22:12:13.253877 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7115,"thread_start_us":130,"threads_started":1}
I20260811 22:12:13.254698 12335 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2aef699d42de4435a2c4bf70ffead64b" candidate_uuid: "cb07dda674f44e7698f0cabffacb3b35" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "713c883b5cf940269f1e1ae888416a75" is_pre_election: true
I20260811 22:12:13.255419 11947 leader_election.cc:302] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: cb07dda674f44e7698f0cabffacb3b35; no voters: 713c883b5cf940269f1e1ae888416a75, 7304252694f14ad5a4732ee0c5824f3d
I20260811 22:12:13.255987 12439 raft_consensus.cc:2726] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260811 22:12:13.262425 11845 catalog_manager.cc:5495] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 reported cstate change: term changed from 0 to 1, leader changed from <none> to 713c883b5cf940269f1e1ae888416a75 (127.28.47.131). New cstate: current_term: 1 leader_uuid: "713c883b5cf940269f1e1ae888416a75" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "cb07dda674f44e7698f0cabffacb3b35" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 36403 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "7304252694f14ad5a4732ee0c5824f3d" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 43721 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "713c883b5cf940269f1e1ae888416a75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 41339 } health_report { overall_health: HEALTHY } } }
I20260811 22:12:13.317509 12430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.311337 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47726 (local address 127.28.47.131:41339)
0811 22:12:13.311505 (+ 168us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:13.311511 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:13.311866 (+ 355us) server_negotiation.cc:407] Connection header received
0811 22:12:13.311975 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:13.311978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:13.312066 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:13.312183 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:13.312592 (+ 409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.313453 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.314800 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:13.315153 (+ 353us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:13.315269 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:13.315771 (+ 502us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:13.316460 (+ 689us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:13.316595 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:13.317251 (+ 656us) server_negotiation.cc:299] Negotiation successful
0811 22:12:13.317322 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:12:13.327304 12175 raft_consensus.cc:1261] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d [term 1 FOLLOWER]: Refusing update from remote peer 713c883b5cf940269f1e1ae888416a75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:13.327481 12002 raft_consensus.cc:1261] T 2aef699d42de4435a2c4bf70ffead64b P cb07dda674f44e7698f0cabffacb3b35 [term 1 FOLLOWER]: Refusing update from remote peer 713c883b5cf940269f1e1ae888416a75: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:13.328225 12437 consensus_queue.cc:1035] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "cb07dda674f44e7698f0cabffacb3b35" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 36403 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:13.329102 12437 consensus_queue.cc:1035] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7304252694f14ad5a4732ee0c5824f3d" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 43721 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:13.350152 12457 mvcc.cc:205] Tried to move back new op lower bound from 7317448021296652288 to 7317448020991692800. Current Snapshot: MvccSnapshot[applied={T|T < 7317448021296652288 or (T in {7317448021296652288})}]
I20260811 22:12:13.351055 12459 mvcc.cc:205] Tried to move back new op lower bound from 7317448021296652288 to 7317448020991692800. Current Snapshot: MvccSnapshot[applied={T|T < 7317448021296652288}]
I20260811 22:12:13.358126 11845 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:13.358359 11845 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46658:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:13.367280 12458 mvcc.cc:205] Tried to move back new op lower bound from 7317448021296652288 to 7317448020991692800. Current Snapshot: MvccSnapshot[applied={T|T < 7317448021296652288}]
I20260811 22:12:13.375558 12305 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aef699d42de4435a2c4bf70ffead64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:13 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:47722
I20260811 22:12:13.377456 12147 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aef699d42de4435a2c4bf70ffead64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:13 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:53618
I20260811 22:12:13.378619 11845 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 413c8b4e69b64ef39d7f9a27167aa686: Sending DeleteTablet for 3 replicas of tablet 2aef699d42de4435a2c4bf70ffead64b
I20260811 22:12:13.379402 11988 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aef699d42de4435a2c4bf70ffead64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:13 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:37662
I20260811 22:12:13.388195 12147 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aef699d42de4435a2c4bf70ffead64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:13 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:53618
I20260811 22:12:13.391378 12305 tablet_service.cc:1514] Processing DeleteTablet for tablet 2aef699d42de4435a2c4bf70ffead64b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:13 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:47722
I20260811 22:12:13.394512 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 11887
I20260811 22:12:13.412561 12469 tablet_replica.cc:330] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d: stopping tablet replica
I20260811 22:12:13.413086 12469 raft_consensus.cc:2229] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:13.413486 12469 raft_consensus.cc:2258] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:13.414007 12468 tablet_replica.cc:330] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75: stopping tablet replica
I20260811 22:12:13.414389 12468 raft_consensus.cc:2229] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:12:13.414809 12468 raft_consensus.cc:2258] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260811 22:12:13.418675 11828 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:36403 (error 108)
I20260811 22:12:13.419520 12469 ts_tablet_manager.cc:1905] T 2aef699d42de4435a2c4bf70ffead64b P 7304252694f14ad5a4732ee0c5824f3d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:13.419581 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 12080
I20260811 22:12:13.422188 12468 ts_tablet_manager.cc:1905] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:13.426649 12425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:13.426221 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:36403 (local address 127.0.0.1:37668)
0811 22:12:13.426409 (+ 188us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:13.426537 (+ 128us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:36403: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 22:12:13.426918 11828 catalog_manager.cc:4584] TS cb07dda674f44e7698f0cabffacb3b35 (127.28.47.129:36403): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2aef699d42de4435a2c4bf70ffead64b: Network error: Client connection negotiation failed: client connection to 127.28.47.129:36403: connect: Connection refused (error 111)
I20260811 22:12:13.436205 12468 ts_tablet_manager.cc:1918] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:13.436365 12468 log.cc:1196] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/2aef699d42de4435a2c4bf70ffead64b
I20260811 22:12:13.436760 12468 ts_tablet_manager.cc:1939] T 2aef699d42de4435a2c4bf70ffead64b P 713c883b5cf940269f1e1ae888416a75: Deleting consensus metadata
W20260811 22:12:13.438962 11830 catalog_manager.cc:4808] TS 713c883b5cf940269f1e1ae888416a75 (127.28.47.131:41339): delete failed for tablet 2aef699d42de4435a2c4bf70ffead64b because the tablet was not found. No further retry: Not found: Tablet not found: 2aef699d42de4435a2c4bf70ffead64b
W20260811 22:12:13.447814 12425 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:13.447048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:43721 (local address 127.0.0.1:53624)
0811 22:12:13.447302 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:13.447320 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:13.447436 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:13.447674 (+ 238us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.130:43721: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":166}
I20260811 22:12:13.448613 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 12233
I20260811 22:12:13.465196 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 11808
I20260811 22:12:13.479650 28862 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260811 22:12:13.826243 28862 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260811 22:12:13.826375 28862 mini_ranger.cc:66] Stopping Ranger...
I20260811 22:12:14.183671 28862 mini_ranger.cc:68] Stopped Ranger
2026-08-11 22:12:14.183 UTC [10645] LOG: received smart shutdown request
2026-08-11 22:12:14.189 UTC [10645] LOG: background worker "logical replication launcher" (PID 10654) exited with exit code 1
2026-08-11 22:12:14.189 UTC [10649] LOG: shutting down
2026-08-11 22:12:14.399 UTC [10645] LOG: database system is shut down
2026-08-11T22:12:14Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (57559 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:12:14 dist-test-slave-vxj2 krb5kdc[12481](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:12:14 dist-test-slave-vxj2 krb5kdc[12481](info): set up 2 sockets
Aug 11 22:12:14 dist-test-slave-vxj2 krb5kdc[12481](info): commencing operation
krb5kdc: starting...
W20260811 22:12:15.700367 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:12:15 dist-test-slave-vxj2 krb5kdc[12481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:12:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:12:15.842730 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:43927
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:35103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:43927,127.28.47.189:39707,127.28.47.188:36193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:15.964313 12498 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:15.964699 12498 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:15.964788 12498 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:15.969084 12498 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:15.969247 12498 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:15.969300 12498 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:15.969340 12498 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:15.969405 12498 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:15.975260 12498 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:35103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:43927,127.28.47.189:39707,127.28.47.188:36193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:43927
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12498
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:15.976544 12498 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:15.977751 12498 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:12:15.987319 12498 server_base.cc:1034] running on GCE node
W20260811 22:12:15.987296 12505 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:15.987298 12507 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:15.987530 12504 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:15.988243 12498 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:15.989668 12498 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:15.990862 12498 hybrid_clock.cc:648] HybridClock initialized: now 1786486335990828 us; error 52 us; skew 500 ppm
Aug 11 22:12:15 dist-test-slave-vxj2 krb5kdc[12481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486335, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:15.994014 12498 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:12:15.995069 12498 webserver.cc:460] Webserver started at http://127.28.47.190:35955/ using document root <none> and password file <none>
I20260811 22:12:15.995688 12498 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:15.995782 12498 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:15.996037 12498 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:15.998386 12498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "09df0d3ebcfb4de68932dc9365a6eba6"
format_stamp: "Formatted at 2026-08-11 22:12:15 on dist-test-slave-vxj2"
server_key: "3e8fbe7ae6d808eb9dba6589bdcc0917"
server_key_iv: "ef913f193aef3b64b5202e09daa77be2"
server_key_version: "encryptionkey@0"
I20260811 22:12:15.999034 12498 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "09df0d3ebcfb4de68932dc9365a6eba6"
format_stamp: "Formatted at 2026-08-11 22:12:15 on dist-test-slave-vxj2"
server_key: "3e8fbe7ae6d808eb9dba6589bdcc0917"
server_key_iv: "ef913f193aef3b64b5202e09daa77be2"
server_key_version: "encryptionkey@0"
I20260811 22:12:16.003532 12498 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:12:16.006525 12514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:16.008030 12498 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:16.008224 12498 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "09df0d3ebcfb4de68932dc9365a6eba6"
format_stamp: "Formatted at 2026-08-11 22:12:15 on dist-test-slave-vxj2"
server_key: "3e8fbe7ae6d808eb9dba6589bdcc0917"
server_key_iv: "ef913f193aef3b64b5202e09daa77be2"
server_key_version: "encryptionkey@0"
I20260811 22:12:16.008376 12498 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:16.035319 12498 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:16.039228 12498 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:16.039507 12498 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:16.048158 12498 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:43927
I20260811 22:12:16.048183 12575 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:43927 every 8 connection(s)
I20260811 22:12:16.049332 12498 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:16.051415 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 12498
I20260811 22:12:16.051533 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:12:16.051901 28862 external_mini_cluster.cc:1428] Setting key 14a59450ccf222c1b7904fa397e6233d
I20260811 22:12:16.052692 12576 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:16.057948 12576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.47.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.189@KRBTEST.COM" created.
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486335, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:16.079243 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.060996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:43927 (local address 127.0.0.1:55872)
0811 22:12:16.061647 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.061698 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:16.062698 (+ 1000us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:16.063834 (+ 1136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:16.063843 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:16.064605 (+ 762us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:16.065262 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.065303 (+ 41us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.066933 (+ 1630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.066941 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.068125 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.068141 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.069309 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.069316 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.069497 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.070178 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:16.070206 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:16.072311 (+ 2105us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:16.075472 (+ 3161us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.075483 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.075508 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.075847 (+ 339us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.076342 (+ 495us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.076345 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.076346 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.076491 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.077539 (+ 1048us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:16.077545 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:16.077695 (+ 150us) client_negotiation.cc:769] Sending connection context
0811 22:12:16.077967 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 22:12:16.078297 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":419,"thread_start_us":207,"threads_started":1}
I20260811 22:12:16.079246 12580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.062034 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55872 (local address 127.28.47.190:43927)
0811 22:12:16.062363 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.062373 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.062413 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:12:16.063108 (+ 695us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.063113 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.063386 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.063584 (+ 198us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.065607 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.066704 (+ 1097us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.068316 (+ 1612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.069044 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.069291 (+ 247us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.072538 (+ 3247us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.072550 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.072570 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.072610 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.075274 (+ 2664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.076024 (+ 750us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.076032 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.076034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.076206 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.076648 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.076651 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.076652 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.077416 (+ 764us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.077538 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.078218 (+ 680us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.078365 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":116,"threads_started":1}
I20260811 22:12:16.082152 12576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:12:16.083086 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.082746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:39707 (local address 127.0.0.1:41932)
0811 22:12:16.082928 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.083008 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:39707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:12:16.083398 12518 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.47.189:39707: connect: Connection refused (error 111)
I20260811 22:12:16.084475 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.084243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:39707 (local address 127.0.0.1:41948)
0811 22:12:16.084353 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.084413 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:39707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:12:16.084715 12576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.189:39707: Network error: Client connection negotiation failed: client connection to 127.28.47.189:39707: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.189.keytab.
Entry for principal kudu/127.28.47.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.189.keytab.
I20260811 22:12:16.088366 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.189:39707
--webserver_interface=127.28.47.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.189.keytab
--principal=kudu/127.28.47.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:35103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:43927,127.28.47.189:39707,127.28.47.188:36193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:12:16.134054 12576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } attempt: 1
I20260811 22:12:16.135083 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.134658 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:39707 (local address 127.0.0.1:41952)
0811 22:12:16.134866 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.134963 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:39707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 22:12:16.136023 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.135719 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:39707 (local address 127.0.0.1:41962)
0811 22:12:16.135866 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.135937 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:39707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 22:12:16.136176 12576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.189:39707: Network error: Client connection negotiation failed: client connection to 127.28.47.189:39707: connect: Connection refused (error 111)
I20260811 22:12:16.204428 12576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } attempt: 2
I20260811 22:12:16.205654 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.205192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:39707 (local address 127.0.0.1:41964)
0811 22:12:16.205384 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.205497 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:39707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 22:12:16.206606 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.206331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:39707 (local address 127.0.0.1:41970)
0811 22:12:16.206468 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.206531 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:39707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:12:16.206758 12576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.189:39707: Network error: Client connection negotiation failed: client connection to 127.28.47.189:39707: connect: Connection refused (error 111)
W20260811 22:12:16.213701 12583 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:16.214030 12583 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:16.214073 12583 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:16.218384 12583 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:16.218469 12583 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:16.218490 12583 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:16.218508 12583 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:16.218523 12583 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:16.224656 12583 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:35103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:43927,127.28.47.189:39707,127.28.47.188:36193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.189.keytab
--principal=kudu/127.28.47.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.189:39707
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.47.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12583
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:16.226027 12583 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:16.227181 12583 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:16.237419 12589 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:16.237504 12592 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:16.237500 12590 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:16.237828 12583 server_base.cc:1034] running on GCE node
I20260811 22:12:16.238866 12583 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:16.240485 12583 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:16.241814 12583 hybrid_clock.cc:648] HybridClock initialized: now 1786486336241796 us; error 44 us; skew 500 ppm
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:16.245538 12583 init.cc:377] Logged in from keytab as kudu/127.28.47.189@KRBTEST.COM (short username kudu)
I20260811 22:12:16.247025 12583 webserver.cc:460] Webserver started at http://127.28.47.189:44023/ using document root <none> and password file <none>
I20260811 22:12:16.247779 12583 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:16.247916 12583 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:16.248217 12583 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:16.251082 12583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/data/instance:
uuid: "0606f24fba30461d80c4031dc99569ea"
format_stamp: "Formatted at 2026-08-11 22:12:16 on dist-test-slave-vxj2"
server_key: "7dd01c78a3161d5cdafcd94537084f62"
server_key_iv: "99f4a3dfeff054f0433b8cadaa6a5f07"
server_key_version: "encryptionkey@0"
I20260811 22:12:16.252112 12583 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/wal/instance:
uuid: "0606f24fba30461d80c4031dc99569ea"
format_stamp: "Formatted at 2026-08-11 22:12:16 on dist-test-slave-vxj2"
server_key: "7dd01c78a3161d5cdafcd94537084f62"
server_key_iv: "99f4a3dfeff054f0433b8cadaa6a5f07"
server_key_version: "encryptionkey@0"
I20260811 22:12:16.257206 12583 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:12:16.260188 12600 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:16.261948 12583 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:16.262145 12583 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/wal
uuid: "0606f24fba30461d80c4031dc99569ea"
format_stamp: "Formatted at 2026-08-11 22:12:16 on dist-test-slave-vxj2"
server_key: "7dd01c78a3161d5cdafcd94537084f62"
server_key_iv: "99f4a3dfeff054f0433b8cadaa6a5f07"
server_key_version: "encryptionkey@0"
I20260811 22:12:16.262295 12583 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:16.285094 12583 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:16.289839 12583 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:16.290141 12583 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:16.298019 12576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } attempt: 3
I20260811 22:12:16.299116 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.298725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:39707 (local address 127.0.0.1:41974)
0811 22:12:16.298923 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.299011 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:39707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 22:12:16.300064 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.299791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:39707 (local address 127.0.0.1:41978)
0811 22:12:16.299944 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.299998 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:39707: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":76}
W20260811 22:12:16.300246 12576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.189:39707: Network error: Client connection negotiation failed: client connection to 127.28.47.189:39707: connect: Connection refused (error 111)
I20260811 22:12:16.302785 12583 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.189:39707
I20260811 22:12:16.302809 12666 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.189:39707 every 8 connection(s)
I20260811 22:12:16.303954 12583 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/data/info.pb
I20260811 22:12:16.307509 12667 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:16.307730 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 12583
I20260811 22:12:16.307880 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-1/wal/instance
I20260811 22:12:16.308183 28862 external_mini_cluster.cc:1428] Setting key 57fa3652893c3776f0d6f36f1d226548
I20260811 22:12:16.312733 12667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.47.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.188@KRBTEST.COM" created.
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.189@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:16.330621 12580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.315413 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55878 (local address 127.28.47.190:43927)
0811 22:12:16.315606 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.315613 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.316590 (+ 977us) server_negotiation.cc:407] Connection header received
0811 22:12:16.317678 (+ 1088us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.317682 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.317752 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.317884 (+ 132us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.320101 (+ 2217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.320746 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.322283 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.322561 (+ 278us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.322697 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.326337 (+ 3640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.326341 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.326343 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.326377 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.328041 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.328729 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.328733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.328734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.328825 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.329259 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.329263 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.329264 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.329496 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.329618 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.330263 (+ 645us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.330428 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:12:16.331588 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.315694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:43927 (local address 127.0.0.1:55878)
0811 22:12:16.316279 (+ 585us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.316340 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:16.317430 (+ 1090us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:16.318260 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:16.318278 (+ 18us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:16.319128 (+ 850us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:16.319777 (+ 649us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.319797 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.320924 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.320927 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.322123 (+ 1196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.322138 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.322767 (+ 629us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.322771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.322988 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.324137 (+ 1149us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:16.324157 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:16.326063 (+ 1906us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:16.328221 (+ 2158us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.328229 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.328253 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.328588 (+ 335us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.328939 (+ 351us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.328943 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.328945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.329124 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.329621 (+ 497us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:16.329631 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:16.329967 (+ 336us) client_negotiation.cc:769] Sending connection context
0811 22:12:16.330184 (+ 217us) client_negotiation.cc:240] Negotiation successful
0811 22:12:16.330593 (+ 409us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":176,"threads_started":1}
I20260811 22:12:16.334035 12667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.189@KRBTEST.COM for kudu/127.28.47.189@KRBTEST.COM
I20260811 22:12:16.346257 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.334797 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:39707 (local address 127.0.0.1:41990)
0811 22:12:16.335019 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.335037 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:16.335145 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:16.335952 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:16.335956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:16.336177 (+ 221us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:16.336356 (+ 179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.336368 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.337623 (+ 1255us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.337625 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.338455 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.338467 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.339409 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.339416 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.339551 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.340189 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:16.340203 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:16.341733 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:16.344073 (+ 2340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.344075 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.344077 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.344435 (+ 358us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.344832 (+ 397us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.344834 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.344836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.344929 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.345677 (+ 748us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:16.345679 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:16.345776 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 22:12:16.345878 (+ 102us) client_negotiation.cc:240] Negotiation successful
0811 22:12:16.346038 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260811 22:12:16.346323 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.335077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41990 (local address 127.28.47.189:39707)
0811 22:12:16.335493 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.335508 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.335538 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:12:16.335580 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.335584 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.335771 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.335894 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.336524 (+ 630us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.337479 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.338615 (+ 1136us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.339204 (+ 589us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.339297 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.341913 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.341926 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.341943 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.341968 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.343933 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.344564 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.344570 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.344572 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.344656 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.345111 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.345114 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.345116 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.345525 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.345721 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.345997 (+ 276us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.346183 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":294,"thread_start_us":202,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.188.keytab.
Entry for principal kudu/127.28.47.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.188.keytab.
I20260811 22:12:16.348634 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.188:36193
--webserver_interface=127.28.47.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.188.keytab
--principal=kudu/127.28.47.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:35103
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:43927,127.28.47.189:39707,127.28.47.188:36193
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:12:16.348773 12667 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:12:16.349691 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.349378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52206)
0811 22:12:16.349529 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.349611 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 22:12:16.349957 12601 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
I20260811 22:12:16.351135 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.350864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52218)
0811 22:12:16.351001 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.351053 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 22:12:16.351436 12667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:36193: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
I20260811 22:12:16.400830 12667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } attempt: 1
I20260811 22:12:16.402045 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.401654 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52222)
0811 22:12:16.401893 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.401960 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
I20260811 22:12:16.402910 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.402653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52228)
0811 22:12:16.402785 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.402839 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260811 22:12:16.403049 12667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:36193: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
I20260811 22:12:16.443492 12576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } attempt: 4
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486335, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for kudu/127.28.47.189@KRBTEST.COM
I20260811 22:12:16.454856 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.444356 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42004 (local address 127.28.47.189:39707)
0811 22:12:16.444522 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.444528 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.444547 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:12:16.444684 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.444688 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.444744 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.444844 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.445495 (+ 651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.446800 (+ 1305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.448132 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.448503 (+ 371us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.448623 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.451126 (+ 2503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.451130 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.451132 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.451160 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.452851 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.453420 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.453424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.453425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.453541 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.453883 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.453885 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.453887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.454116 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.454215 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.454592 (+ 377us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.454712 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:12:16.454862 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.444209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:39707 (local address 127.0.0.1:42004)
0811 22:12:16.444465 (+ 256us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.444481 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:16.444575 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:16.444947 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:16.444950 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:16.445211 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:16.445359 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.445365 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.446936 (+ 1571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.446939 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.447990 (+ 1051us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.448001 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.448641 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.448644 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.448730 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.449351 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:16.449364 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:16.450938 (+ 1574us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:16.452983 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.452987 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.452989 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.453299 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.453657 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.453660 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.453662 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.453767 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.454227 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:16.454232 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:16.454358 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 22:12:16.454474 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 22:12:16.454647 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138}
I20260811 22:12:16.455807 12576 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:12:16.456673 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.456379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52230)
0811 22:12:16.456524 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.456588 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:12:16.457621 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.457379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52232)
0811 22:12:16.457499 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.457555 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 22:12:16.457813 12576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:36193: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
I20260811 22:12:16.471251 12667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } attempt: 2
I20260811 22:12:16.472271 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.471937 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52234)
0811 22:12:16.472107 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.472188 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 22:12:16.473135 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.472890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52246)
0811 22:12:16.473014 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.473072 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:12:16.473378 12667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:36193: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
W20260811 22:12:16.480670 12674 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:16.481005 12674 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:16.481052 12674 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:16.485452 12674 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:16.485541 12674 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:16.485561 12674 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:16.485579 12674 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:16.485594 12674 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:16.491991 12674 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:35103
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:43927,127.28.47.189:39707,127.28.47.188:36193
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.188.keytab
--principal=kudu/127.28.47.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.188:36193
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.47.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12674
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:16.493459 12674 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:16.494670 12674 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:16.504959 12682 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:16.505118 12684 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:16.505182 12674 server_base.cc:1034] running on GCE node
W20260811 22:12:16.505456 12680 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:16.506135 12674 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:16.507412 12674 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:16.508611 12674 hybrid_clock.cc:648] HybridClock initialized: now 1786486336508596 us; error 31 us; skew 500 ppm
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:16.512363 12674 init.cc:377] Logged in from keytab as kudu/127.28.47.188@KRBTEST.COM (short username kudu)
I20260811 22:12:16.513662 12674 webserver.cc:460] Webserver started at http://127.28.47.188:40139/ using document root <none> and password file <none>
I20260811 22:12:16.514264 12674 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:16.514333 12674 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:16.514523 12674 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:16.517004 12576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } attempt: 1
I20260811 22:12:16.516745 12674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/data/instance:
uuid: "016f36097ca94524a9b8e4e921d156bd"
format_stamp: "Formatted at 2026-08-11 22:12:16 on dist-test-slave-vxj2"
server_key: "9701ee2696218b0d8e1539f85ac12bae"
server_key_iv: "86bb1a6f9a79ce804897251e82966f51"
server_key_version: "encryptionkey@0"
I20260811 22:12:16.517490 12674 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/wal/instance:
uuid: "016f36097ca94524a9b8e4e921d156bd"
format_stamp: "Formatted at 2026-08-11 22:12:16 on dist-test-slave-vxj2"
server_key: "9701ee2696218b0d8e1539f85ac12bae"
server_key_iv: "86bb1a6f9a79ce804897251e82966f51"
server_key_version: "encryptionkey@0"
I20260811 22:12:16.517925 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.517578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52254)
0811 22:12:16.517760 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.517830 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260811 22:12:16.518762 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.518494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52268)
0811 22:12:16.518615 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.518682 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 22:12:16.518916 12576 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:36193: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
I20260811 22:12:16.522315 12674 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:12:16.525293 12691 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:16.526798 12674 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:16.527119 12674 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/wal
uuid: "016f36097ca94524a9b8e4e921d156bd"
format_stamp: "Formatted at 2026-08-11 22:12:16 on dist-test-slave-vxj2"
server_key: "9701ee2696218b0d8e1539f85ac12bae"
server_key_iv: "86bb1a6f9a79ce804897251e82966f51"
server_key_version: "encryptionkey@0"
I20260811 22:12:16.527264 12674 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:16.551802 12674 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:16.556818 12674 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:16.557085 12674 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:16.564572 12667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } attempt: 3
I20260811 22:12:16.565850 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.565436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52270)
0811 22:12:16.565641 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.565729 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
I20260811 22:12:16.567148 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.566743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52286)
0811 22:12:16.566939 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.567018 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
W20260811 22:12:16.567384 12667 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:36193: Network error: Client connection negotiation failed: client connection to 127.28.47.188:36193: connect: Connection refused (error 111)
I20260811 22:12:16.570495 12674 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.188:36193
I20260811 22:12:16.570524 12750 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.188:36193 every 8 connection(s)
I20260811 22:12:16.572117 12674 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/data/info.pb
I20260811 22:12:16.575856 12751 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:16.578524 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 12674
I20260811 22:12:16.578673 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786486147647525-28862-0/minicluster-data/master-2/wal/instance
I20260811 22:12:16.578974 28862 external_mini_cluster.cc:1428] Setting key bd2bc40cbc0ba127a43f13d270eb0184
I20260811 22:12:16.581470 12751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:12:16.586566 12576 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } attempt: 2
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.188@KRBTEST.COM
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486335, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for kudu/127.28.47.188@KRBTEST.COM
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.188@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:16.602190 12580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.586018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55890 (local address 127.28.47.190:43927)
0811 22:12:16.586199 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.586205 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.586224 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:12:16.586277 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.586281 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.586340 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.586433 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.588598 (+ 2165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.589519 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.591076 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.591504 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.591605 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.595696 (+ 4091us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.595700 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.595703 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.595733 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.599085 (+ 3352us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.599741 (+ 656us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.599748 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.599753 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.599878 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.600320 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.600326 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.600349 (+ 23us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.600708 (+ 359us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.600843 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.601751 (+ 908us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.601943 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:12:16.604013 12754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.580502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52300 (local address 127.28.47.188:36193)
0811 22:12:16.581101 (+ 599us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.581137 (+ 36us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.581211 (+ 74us) server_negotiation.cc:407] Connection header received
0811 22:12:16.582549 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.582585 (+ 36us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.583109 (+ 524us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.583497 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.584448 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.586296 (+ 1848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.587620 (+ 1324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.588382 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.588548 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.591819 (+ 3271us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.591827 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.591843 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.591876 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.599010 (+ 7134us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.599769 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.599779 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.599781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.599962 (+ 181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.600895 (+ 933us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.600900 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.600904 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.601382 (+ 478us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.601582 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.601884 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.602647 (+ 763us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":384,"thread_start_us":199,"threads_started":1}
I20260811 22:12:16.604637 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.584932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:43927 (local address 127.0.0.1:55890)
0811 22:12:16.585443 (+ 511us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.585496 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:16.585617 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:16.587595 (+ 1978us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:16.587605 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:16.588203 (+ 598us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:16.588436 (+ 233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.588447 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.589698 (+ 1251us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.589704 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.590916 (+ 1212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.590932 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.591821 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.591826 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.591976 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.593061 (+ 1085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:16.593084 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:16.595527 (+ 2443us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:16.599198 (+ 3671us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.599211 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.599225 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.599592 (+ 367us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.600024 (+ 432us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.600029 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.600034 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.600214 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.600841 (+ 627us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:16.600852 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:16.601408 (+ 556us) client_negotiation.cc:769] Sending connection context
0811 22:12:16.602465 (+ 1057us) client_negotiation.cc:240] Negotiation successful
0811 22:12:16.602714 (+ 249us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"mutex_wait_us":23,"thread_start_us":207,"threads_started":1}
I20260811 22:12:16.605041 12579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.587026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52304)
0811 22:12:16.587155 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.587171 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:16.587246 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:16.589050 (+ 1804us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:16.589058 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:16.589308 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:16.589433 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.589441 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.590738 (+ 1297us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.590743 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.591586 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.591597 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.592315 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.592320 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.592429 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.592877 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:16.592898 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:16.594642 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:16.603285 (+ 8643us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.603290 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.603292 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.603515 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.603860 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.603863 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.603865 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.603954 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.604535 (+ 581us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:16.604538 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:16.604643 (+ 105us) client_negotiation.cc:769] Sending connection context
0811 22:12:16.604726 (+ 83us) client_negotiation.cc:240] Negotiation successful
0811 22:12:16.604891 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38}
I20260811 22:12:16.606400 12756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.587295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52304 (local address 127.28.47.188:36193)
0811 22:12:16.588775 (+ 1480us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.588786 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.588812 (+ 26us) server_negotiation.cc:407] Connection header received
0811 22:12:16.588853 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.588857 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.588904 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.589557 (+ 653us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.589672 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.590588 (+ 916us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.591813 (+ 1225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.592172 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.592650 (+ 478us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.594848 (+ 2198us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.594855 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.594921 (+ 66us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.594956 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.603158 (+ 8202us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.603630 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.603634 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.603635 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.603764 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.604049 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.604053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.604055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.604413 (+ 358us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.605954 (+ 1541us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.606044 (+ 90us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.606203 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":1404,"thread_start_us":129,"threads_started":1}
I20260811 22:12:16.607215 12751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:12:16.610702 12576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6: Bootstrap starting.
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.188@KRBTEST.COM for kudu/127.28.47.189@KRBTEST.COM
I20260811 22:12:16.614115 12576 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:16.615197 12576 log.cc:824] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:16.617276 12580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.607282 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55894 (local address 127.28.47.190:43927)
0811 22:12:16.607483 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.607488 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.607503 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:12:16.607541 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.607545 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.607605 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.607722 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.608604 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.609123 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.610016 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.610253 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.610375 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.613303 (+ 2928us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.613307 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.613309 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.613383 (+ 74us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.615225 (+ 1842us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.615794 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.615798 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.615800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.615889 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.616282 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.616286 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.616288 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.616551 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.616731 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.616953 (+ 222us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.617079 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260811 22:12:16.617929 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.608114 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42016 (local address 127.28.47.189:39707)
0811 22:12:16.608269 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.608273 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.608287 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:12:16.608331 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.608334 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.608378 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.608471 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.608978 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.609724 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.610984 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.611293 (+ 309us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.611371 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.613875 (+ 2504us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.613882 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.613887 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.613923 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.615649 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.616140 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.616145 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.616147 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.616246 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.616571 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.616574 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.616576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.616811 (+ 235us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.616906 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.617643 (+ 737us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.617774 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:12:16.617957 12576 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6: No bootstrap required, opened a new log
I20260811 22:12:16.618424 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.607756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:39707 (local address 127.0.0.1:42016)
0811 22:12:16.608066 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.608082 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:16.608159 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:16.608512 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:16.608514 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:16.608687 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:16.608821 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.608829 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.609892 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.609895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.610825 (+ 930us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.610838 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.611540 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.611544 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.611707 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.612211 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:16.612246 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:16.613699 (+ 1453us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:16.615799 (+ 2100us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.615802 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.615805 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.616020 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.616355 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.616359 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.616361 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.616457 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.616957 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:16.616960 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:16.617075 (+ 115us) client_negotiation.cc:769] Sending connection context
0811 22:12:16.618076 (+ 1001us) client_negotiation.cc:240] Negotiation successful
0811 22:12:16.618241 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227}
I20260811 22:12:16.619244 12751 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:12:16.622774 12576 raft_consensus.cc:348] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } }
I20260811 22:12:16.623021 12576 raft_consensus.cc:374] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:16.623086 12576 raft_consensus.cc:729] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09df0d3ebcfb4de68932dc9365a6eba6, State: Initialized, Role: FOLLOWER
I20260811 22:12:16.623672 12576 consensus_queue.cc:260] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } }
I20260811 22:12:16.624464 12758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } } }
I20260811 22:12:16.624771 12758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.188@KRBTEST.COM for kudu/127.28.47.188@KRBTEST.COM
I20260811 22:12:16.625638 12576 sys_catalog.cc:564] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486335, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.189@KRBTEST.COM
I20260811 22:12:16.630630 12755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.619769 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52312)
0811 22:12:16.620016 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.620029 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:16.620117 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:16.620381 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:16.620384 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:16.620566 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:16.620668 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.620675 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.621770 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.621773 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.622572 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.622584 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.623294 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.623298 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.623384 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.623882 (+ 498us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:16.623897 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:16.626094 (+ 2197us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:16.628662 (+ 2568us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.628665 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.628669 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.628960 (+ 291us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.629353 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.629359 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.629363 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.629453 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.630033 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:16.630038 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:16.630151 (+ 113us) client_negotiation.cc:769] Sending connection context
0811 22:12:16.630260 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 22:12:16.630446 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197}
I20260811 22:12:16.631742 12756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.619837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52312 (local address 127.28.47.188:36193)
0811 22:12:16.619951 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.619956 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.620106 (+ 150us) server_negotiation.cc:407] Connection header received
0811 22:12:16.620209 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.620215 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.620266 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.620337 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.620797 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.621621 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.622733 (+ 1112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.623142 (+ 409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.623248 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.626296 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.626300 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.626302 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.626355 (+ 53us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.628505 (+ 2150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.629084 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.629089 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.629091 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.629248 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.629547 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.629552 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.629555 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.629898 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.629997 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.631440 (+ 1443us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.631569 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:12:16.632110 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.618573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42020 (local address 127.28.47.189:39707)
0811 22:12:16.618943 (+ 370us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.618947 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.619123 (+ 176us) server_negotiation.cc:407] Connection header received
0811 22:12:16.619261 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.619264 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.619307 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.619404 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.620249 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.620879 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.621978 (+ 1099us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.622209 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.622329 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.627308 (+ 4979us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.627341 (+ 33us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.627345 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.627385 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.629288 (+ 1903us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.629903 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.629908 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.629909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.630008 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.630347 (+ 339us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.630350 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.630351 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.630582 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.630892 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.631815 (+ 923us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.631961 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":290}
W20260811 22:12:16.633390 12773 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:12:16.633476 12773 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:12:16.635119 12751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd: Bootstrap starting.
I20260811 22:12:16.636332 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:12:16.632436 12498 master_runner.cc:186] Error getting master registration for 127.28.47.189:39707: OK, instance_id { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" instance_seqno: 1786486336299134 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:12:16.637375 12498 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:12:16.638199 12751 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:16.639206 12751 log.cc:824] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:16.643639 12751 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd: No bootstrap required, opened a new log
I20260811 22:12:16.649331 12751 raft_consensus.cc:348] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } }
I20260811 22:12:16.649700 12751 raft_consensus.cc:374] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:16.649799 12751 raft_consensus.cc:729] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 016f36097ca94524a9b8e4e921d156bd, State: Initialized, Role: FOLLOWER
I20260811 22:12:16.650239 12756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.639064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52318 (local address 127.28.47.188:36193)
0811 22:12:16.639196 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.639201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.639452 (+ 251us) server_negotiation.cc:407] Connection header received
0811 22:12:16.639658 (+ 206us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.639662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.639713 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.639771 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.640891 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.641762 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.642880 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.643170 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.643248 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.645589 (+ 2341us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.645593 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.645596 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.645627 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.647762 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.648509 (+ 747us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.648513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.648515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.648618 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.648956 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.648959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.648961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.649261 (+ 300us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.649433 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.649936 (+ 503us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.650083 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:12:16.650282 12580 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.638752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55898 (local address 127.28.47.190:43927)
0811 22:12:16.638965 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.638970 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.640131 (+ 1161us) server_negotiation.cc:407] Connection header received
0811 22:12:16.640316 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.640319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.640371 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.640425 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.641502 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.642282 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.643509 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.643861 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.643945 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.645807 (+ 1862us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.645811 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.645827 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.645862 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.647699 (+ 1837us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.648513 (+ 814us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.648517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.648519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.648622 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.649015 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.649019 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.649021 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.649309 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.649431 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.649947 (+ 516us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.650111 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":126}
I20260811 22:12:16.650413 12751 consensus_queue.cc:260] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } }
I20260811 22:12:16.652169 12751 sys_catalog.cc:564] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:16.653493 12789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } } }
I20260811 22:12:16.653716 12789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [sys.catalog]: This master's current role is: FOLLOWER
W20260811 22:12:16.660485 12801 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:12:16.660598 12801 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:12:16.662771 12672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.639503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42034 (local address 127.28.47.189:39707)
0811 22:12:16.639632 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.639636 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.640151 (+ 515us) server_negotiation.cc:407] Connection header received
0811 22:12:16.640870 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.640874 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.640927 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.640979 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.653446 (+ 12467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.654276 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.655802 (+ 1526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.656145 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.656280 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.658289 (+ 2009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.658293 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.658296 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.658337 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.660188 (+ 1851us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.660771 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.660775 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.660777 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.660886 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.661261 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.661265 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.661266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.661515 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.661661 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.662437 (+ 776us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.662569 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:12:16.663360 12674 master_runner.cc:186] Error getting master registration for 127.28.47.189:39707: OK, instance_id { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" instance_seqno: 1786486336299134 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:12:16.667110 12674 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:12:16.710613 12667 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } attempt: 4
Aug 11 22:12:16 dist-test-slave-vxj2 krb5kdc[12481](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486336, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.189@KRBTEST.COM for kudu/127.28.47.188@KRBTEST.COM
I20260811 22:12:16.722391 12670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.711329 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:36193 (local address 127.0.0.1:52326)
0811 22:12:16.711584 (+ 255us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:16.711601 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:16.711707 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:16.712060 (+ 353us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:16.712063 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:16.712302 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:16.712427 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.712434 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.713589 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.713594 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.714511 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:16.714525 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.715290 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.715294 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:16.715390 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.716008 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:16.716025 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:16.717857 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:16.720341 (+ 2484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.720346 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.720348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.720693 (+ 345us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.721083 (+ 390us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:16.721087 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:16.721089 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:16.721248 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:16.721762 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:16.721766 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:16.721892 (+ 126us) client_negotiation.cc:769] Sending connection context
0811 22:12:16.722014 (+ 122us) client_negotiation.cc:240] Negotiation successful
0811 22:12:16.722193 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167}
I20260811 22:12:16.722432 12756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:16.711435 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52326 (local address 127.28.47.188:36193)
0811 22:12:16.711586 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:16.711591 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:16.711707 (+ 116us) server_negotiation.cc:407] Connection header received
0811 22:12:16.711849 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:16.711853 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:16.711910 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:16.712018 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:16.712576 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.713451 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.714669 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:16.715037 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:16.715140 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:16.718072 (+ 2932us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:16.718077 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:16.718080 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:16.718114 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:16.720155 (+ 2041us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.720849 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.720853 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.720855 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.720961 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:16.721361 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:16.721365 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:16.721367 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:16.721640 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:16.721751 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:16.722096 (+ 345us) server_negotiation.cc:299] Negotiation successful
0811 22:12:16.722249 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:12:16.726939 12667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea: Bootstrap starting.
I20260811 22:12:16.730427 12667 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:16.731541 12667 log.cc:824] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:16.734619 12667 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea: No bootstrap required, opened a new log
I20260811 22:12:16.739400 12667 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } }
I20260811 22:12:16.739734 12667 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:16.739790 12667 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0606f24fba30461d80c4031dc99569ea, State: Initialized, Role: FOLLOWER
I20260811 22:12:16.740378 12667 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } }
I20260811 22:12:16.741053 12805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } } }
I20260811 22:12:16.741328 12805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:12:16.742419 12667 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:12:16.749965 12822 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:12:16.750097 12822 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:12:16.753343 12583 master_runner.cc:263] No leader master found from master 0606f24fba30461d80c4031dc99569ea
I20260811 22:12:16.753458 12583 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 22:12:16.791708 12520 tablet.cc:2369] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:12:16.792479 12610 tablet.cc:2369] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:12:16.809365 12698 tablet.cc:2369] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:12:16.824960 12805 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:16.825246 12805 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } }
I20260811 22:12:16.826467 12805 leader_election.cc:288] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 09df0d3ebcfb4de68932dc9365a6eba6 (127.28.47.190:43927), 016f36097ca94524a9b8e4e921d156bd (127.28.47.188:36193)
I20260811 22:12:16.827136 12545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0606f24fba30461d80c4031dc99569ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" is_pre_election: true
I20260811 22:12:16.827092 12720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0606f24fba30461d80c4031dc99569ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016f36097ca94524a9b8e4e921d156bd" is_pre_election: true
I20260811 22:12:16.827441 12545 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0606f24fba30461d80c4031dc99569ea in term 0.
I20260811 22:12:16.827442 12720 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0606f24fba30461d80c4031dc99569ea in term 0.
I20260811 22:12:16.827970 12601 leader_election.cc:302] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016f36097ca94524a9b8e4e921d156bd, 0606f24fba30461d80c4031dc99569ea; no voters:
I20260811 22:12:16.828246 12805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:16.828325 12805 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:16.828377 12805 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:16.829730 12805 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } }
I20260811 22:12:16.830147 12805 leader_election.cc:288] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [CANDIDATE]: Term 1 election: Requested vote from peers 09df0d3ebcfb4de68932dc9365a6eba6 (127.28.47.190:43927), 016f36097ca94524a9b8e4e921d156bd (127.28.47.188:36193)
I20260811 22:12:16.830538 12545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0606f24fba30461d80c4031dc99569ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "09df0d3ebcfb4de68932dc9365a6eba6"
I20260811 22:12:16.830610 12720 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0606f24fba30461d80c4031dc99569ea" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "016f36097ca94524a9b8e4e921d156bd"
I20260811 22:12:16.830703 12545 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:16.830780 12720 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:16.832110 12720 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0606f24fba30461d80c4031dc99569ea in term 1.
I20260811 22:12:16.832110 12545 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0606f24fba30461d80c4031dc99569ea in term 1.
I20260811 22:12:16.832504 12601 leader_election.cc:302] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 016f36097ca94524a9b8e4e921d156bd, 0606f24fba30461d80c4031dc99569ea; no voters:
I20260811 22:12:16.832724 12805 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:16.833029 12805 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [term 1 LEADER]: Becoming Leader. State: Replica: 0606f24fba30461d80c4031dc99569ea, State: Running, Role: LEADER
I20260811 22:12:16.833631 12805 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } }
I20260811 22:12:16.835811 12826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0606f24fba30461d80c4031dc99569ea. Latest consensus state: current_term: 1 leader_uuid: "0606f24fba30461d80c4031dc99569ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } } }
I20260811 22:12:16.835949 12826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:16.836402 12828 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:16.837469 12828 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:16.843762 12720 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [term 1 FOLLOWER]: Refusing update from remote peer 0606f24fba30461d80c4031dc99569ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:16.844197 12545 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [term 1 FOLLOWER]: Refusing update from remote peer 0606f24fba30461d80c4031dc99569ea: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:16.844528 12805 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:16.845027 12826 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [LEADER]: Connected to new peer: Peer: permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:16.849264 12789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0606f24fba30461d80c4031dc99569ea. Latest consensus state: current_term: 1 leader_uuid: "0606f24fba30461d80c4031dc99569ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } } }
I20260811 22:12:16.849236 12758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0606f24fba30461d80c4031dc99569ea. Latest consensus state: current_term: 1 leader_uuid: "0606f24fba30461d80c4031dc99569ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } } }
I20260811 22:12:16.849404 12758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:12:16.849404 12789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:12:16.852671 12826 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0606f24fba30461d80c4031dc99569ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } } }
I20260811 22:12:16.852877 12826 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:16.854589 12789 sys_catalog.cc:455] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0606f24fba30461d80c4031dc99569ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } } }
I20260811 22:12:16.854771 12789 sys_catalog.cc:458] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:12:16.855669 12829 mvcc.cc:205] Tried to move back new op lower bound from 7317448035700011008 to 7317448035673812992. Current Snapshot: MvccSnapshot[applied={T|T < 7317448035700011008}]
I20260811 22:12:16.856240 12831 mvcc.cc:205] Tried to move back new op lower bound from 7317448035700011008 to 7317448035673812992. Current Snapshot: MvccSnapshot[applied={T|T < 7317448035700011008}]
I20260811 22:12:16.857064 12805 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0606f24fba30461d80c4031dc99569ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } } }
I20260811 22:12:16.857257 12805 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:16.858777 12758 sys_catalog.cc:455] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0606f24fba30461d80c4031dc99569ea" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "09df0d3ebcfb4de68932dc9365a6eba6" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43927 } } peers { permanent_uuid: "0606f24fba30461d80c4031dc99569ea" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 39707 } } peers { permanent_uuid: "016f36097ca94524a9b8e4e921d156bd" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 36193 } } }
I20260811 22:12:16.858948 12758 sys_catalog.cc:458] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:12:16.861877 12830 mvcc.cc:205] Tried to move back new op lower bound from 7317448035700011008 to 7317448035673812992. Current Snapshot: MvccSnapshot[applied={T|T < 7317448035700011008 or (T in {7317448035700011008})}]
I20260811 22:12:16.863925 12828 catalog_manager.cc:1347] Generated new cluster ID: b769c440ce4a458ca2ebbc6aa84e9261
I20260811 22:12:16.864020 12828 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:16.876163 12828 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:16.877525 12828 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:16.885970 12828 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0606f24fba30461d80c4031dc99569ea: Generated new TSK 0
I20260811 22:12:16.886988 12828 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:12:17.634934 12773 catalog_manager.cc:1259] Loaded cluster ID: b769c440ce4a458ca2ebbc6aa84e9261
I20260811 22:12:17.635080 12773 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6: loading cluster ID for follower catalog manager: success
I20260811 22:12:17.637099 12773 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6: acquiring CA information for follower catalog manager: success
I20260811 22:12:17.638600 12773 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 09df0d3ebcfb4de68932dc9365a6eba6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:12:17.662468 12801 catalog_manager.cc:1259] Loaded cluster ID: b769c440ce4a458ca2ebbc6aa84e9261
I20260811 22:12:17.662581 12801 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd: loading cluster ID for follower catalog manager: success
I20260811 22:12:17.664350 12801 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd: acquiring CA information for follower catalog manager: success
I20260811 22:12:17.665697 12801 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 016f36097ca94524a9b8e4e921d156bd: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:12:17.968619 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 12498
I20260811 22:12:17.979638 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 12583
I20260811 22:12:17.990505 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 12674
2026-08-11T22:12:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3459 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:12:18 dist-test-slave-vxj2 krb5kdc[12845](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:12:18 dist-test-slave-vxj2 krb5kdc[12845](info): set up 2 sockets
Aug 11 22:12:18 dist-test-slave-vxj2 krb5kdc[12845](info): commencing operation
krb5kdc: starting...
W20260811 22:12:19.176267 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.158s user 0.004s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:12:19 dist-test-slave-vxj2 krb5kdc[12845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:12:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:12:19.318413 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45017
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:46707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:45017
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:19.445550 12860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:19.445924 12860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:19.446017 12860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:19.450291 12860 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:19.450423 12860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:19.450464 12860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:19.450503 12860 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:19.450539 12860 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:19.456290 12860 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:46707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:45017
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:19.457650 12860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:19.458879 12860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:19.468534 12865 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:19.468531 12869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:19.468649 12866 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:19.468746 12860 server_base.cc:1034] running on GCE node
I20260811 22:12:19.469650 12860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:19.470983 12860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:19.472250 12860 hybrid_clock.cc:648] HybridClock initialized: now 1786486339472221 us; error 44 us; skew 500 ppm
Aug 11 22:12:19 dist-test-slave-vxj2 krb5kdc[12845](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486339, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:19.475700 12860 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:12:19.476923 12860 webserver.cc:460] Webserver started at http://127.28.47.190:46399/ using document root <none> and password file <none>
I20260811 22:12:19.477667 12860 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:19.477785 12860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:19.478063 12860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:19.480463 12860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "55665d61862b4704899678b35db3c92f"
format_stamp: "Formatted at 2026-08-11 22:12:19 on dist-test-slave-vxj2"
server_key: "17fb66d53cfb4991f98b8b9c797ede10"
server_key_iv: "70e545cf90a2753a7cc0d0849448b273"
server_key_version: "encryptionkey@0"
I20260811 22:12:19.481218 12860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "55665d61862b4704899678b35db3c92f"
format_stamp: "Formatted at 2026-08-11 22:12:19 on dist-test-slave-vxj2"
server_key: "17fb66d53cfb4991f98b8b9c797ede10"
server_key_iv: "70e545cf90a2753a7cc0d0849448b273"
server_key_version: "encryptionkey@0"
I20260811 22:12:19.486574 12860 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.001s
I20260811 22:12:19.489526 12876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:19.491257 12860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:19.491462 12860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "55665d61862b4704899678b35db3c92f"
format_stamp: "Formatted at 2026-08-11 22:12:19 on dist-test-slave-vxj2"
server_key: "17fb66d53cfb4991f98b8b9c797ede10"
server_key_iv: "70e545cf90a2753a7cc0d0849448b273"
server_key_version: "encryptionkey@0"
I20260811 22:12:19.491618 12860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:19.508208 12860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:19.512063 12860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:19.512353 12860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:19.521219 12860 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:45017
I20260811 22:12:19.521255 12939 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:45017 every 8 connection(s)
I20260811 22:12:19.522468 12860 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:19.525918 12940 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:19.526890 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 12860
I20260811 22:12:19.527026 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:12:19.527336 28862 external_mini_cluster.cc:1428] Setting key 3dd14cff16d163bbd3a1a1b65354f43a
I20260811 22:12:19.533581 12940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f: Bootstrap starting.
I20260811 22:12:19.536633 12940 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f: Neither blocks nor log segments found. Creating new log.
Aug 11 22:12:19 dist-test-slave-vxj2 krb5kdc[12845](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486339, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:19.537595 12940 log.cc:824] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:19.539709 12940 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f: No bootstrap required, opened a new log
I20260811 22:12:19.543285 12943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:19.528888 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47986 (local address 127.28.47.190:45017)
0811 22:12:19.529406 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:19.529425 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:19.529468 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:12:19.530168 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:19.530186 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:19.530570 (+ 384us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:19.530886 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:19.531941 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:19.533024 (+ 1083us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:19.534352 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:19.534884 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:19.535014 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:19.538133 (+ 3119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:19.538139 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:19.538154 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:19.538194 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:19.540135 (+ 1941us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:19.540643 (+ 508us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:19.540649 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:19.540654 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:19.540779 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:19.541108 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:19.541111 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:19.541112 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:19.541602 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:19.541842 (+ 240us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:19.542151 (+ 309us) server_negotiation.cc:299] Negotiation successful
0811 22:12:19.542477 (+ 326us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":200,"threads_started":1}
I20260811 22:12:19.543692 12940 raft_consensus.cc:348] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55665d61862b4704899678b35db3c92f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45017 } }
I20260811 22:12:19.543977 12940 raft_consensus.cc:374] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:19.544061 12940 raft_consensus.cc:729] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55665d61862b4704899678b35db3c92f, State: Initialized, Role: FOLLOWER
I20260811 22:12:19.544605 12940 consensus_queue.cc:260] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55665d61862b4704899678b35db3c92f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45017 } }
I20260811 22:12:19.544781 12940 raft_consensus.cc:388] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:19.544859 12940 raft_consensus.cc:482] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:19.544962 12940 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:19.546332 12940 raft_consensus.cc:504] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55665d61862b4704899678b35db3c92f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45017 } }
I20260811 22:12:19.546757 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:12:19.546772 12940 leader_election.cc:302] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 55665d61862b4704899678b35db3c92f; no voters:
I20260811 22:12:19.547152 12940 leader_election.cc:288] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:19.547297 12945 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:19.547544 12945 raft_consensus.cc:686] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [term 1 LEADER]: Becoming Leader. State: Replica: 55665d61862b4704899678b35db3c92f, State: Running, Role: LEADER
I20260811 22:12:19.547976 12945 consensus_queue.cc:237] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55665d61862b4704899678b35db3c92f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45017 } }
I20260811 22:12:19.548656 12940 sys_catalog.cc:564] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:19.550042 12947 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 55665d61862b4704899678b35db3c92f. Latest consensus state: current_term: 1 leader_uuid: "55665d61862b4704899678b35db3c92f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55665d61862b4704899678b35db3c92f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45017 } } }
I20260811 22:12:19.550056 12946 sys_catalog.cc:455] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "55665d61862b4704899678b35db3c92f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "55665d61862b4704899678b35db3c92f" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45017 } } }
I20260811 22:12:19.550203 12947 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:19.550204 12946 sys_catalog.cc:458] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:19.550640 12956 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:19.553874 12956 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:19.559762 12956 catalog_manager.cc:1347] Generated new cluster ID: a00c2aa8e7ee433da0a04919c549657b
I20260811 22:12:19.559870 12956 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:19.576815 12956 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:19.578358 12956 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:19.587039 12956 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 55665d61862b4704899678b35db3c92f: Generated new TSK 0
I20260811 22:12:19.588099 12956 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:12:19.603559 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 12860
2026-08-11T22:12:19Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1610 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:12:19 dist-test-slave-vxj2 krb5kdc[12974](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:12:19 dist-test-slave-vxj2 krb5kdc[12974](info): set up 2 sockets
Aug 11 22:12:19 dist-test-slave-vxj2 krb5kdc[12974](info): commencing operation
krb5kdc: starting...
W20260811 22:12:20.758572 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:12:20 dist-test-slave-vxj2 krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:12:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:12:20.901134 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:36885
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:36885
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:21.024358 12988 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:21.024751 12988 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:21.024833 12988 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:21.029295 12988 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:21.029433 12988 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:21.029470 12988 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:21.029510 12988 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:21.029546 12988 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:21.035387 12988 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:36885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:36885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12988
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:21.036724 12988 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:21.037941 12988 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:21.047472 12996 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:21.047429 12994 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:21.047429 12993 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:21.047550 12988 server_base.cc:1034] running on GCE node
I20260811 22:12:21.048491 12988 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:21.049934 12988 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:21.051173 12988 hybrid_clock.cc:648] HybridClock initialized: now 1786486341051151 us; error 37 us; skew 500 ppm
Aug 11 22:12:21 dist-test-slave-vxj2 krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:21.054441 12988 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:12:21.055490 12988 webserver.cc:460] Webserver started at http://127.28.47.190:46509/ using document root <none> and password file <none>
I20260811 22:12:21.056139 12988 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:21.056212 12988 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:21.056399 12988 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:21.058784 12988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "805a0aed16e246bc8c7228ea66c2d766"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "7235c6e74a54b7c367e4e87a2759d12d"
server_key_iv: "b31411d801e9ee364fd8f478cbaedca1"
server_key_version: "encryptionkey@0"
I20260811 22:12:21.059391 12988 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "805a0aed16e246bc8c7228ea66c2d766"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "7235c6e74a54b7c367e4e87a2759d12d"
server_key_iv: "b31411d801e9ee364fd8f478cbaedca1"
server_key_version: "encryptionkey@0"
I20260811 22:12:21.063802 12988 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260811 22:12:21.066886 13003 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:21.068426 12988 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:12:21.068569 12988 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "805a0aed16e246bc8c7228ea66c2d766"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "7235c6e74a54b7c367e4e87a2759d12d"
server_key_iv: "b31411d801e9ee364fd8f478cbaedca1"
server_key_version: "encryptionkey@0"
I20260811 22:12:21.068768 12988 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:21.095009 12988 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:21.099040 12988 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:21.099253 12988 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:21.108268 12988 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:36885
I20260811 22:12:21.108623 13064 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:36885 every 8 connection(s)
I20260811 22:12:21.109527 12988 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:21.109866 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 12988
I20260811 22:12:21.110031 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:12:21.110404 28862 external_mini_cluster.cc:1428] Setting key 581feccd607e9de94dcec2500d73fb07
I20260811 22:12:21.113492 13065 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:21.120215 13065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Bootstrap starting.
Aug 11 22:12:21 dist-test-slave-vxj2 krb5kdc[12974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486340, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:21.123090 13065 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:21.124094 13065 log.cc:824] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:21.126410 13065 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: No bootstrap required, opened a new log
I20260811 22:12:21.128535 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.112065 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57492 (local address 127.28.47.190:36885)
0811 22:12:21.112633 (+ 568us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:21.112650 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:21.112695 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:12:21.113858 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:21.114345 (+ 487us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:21.114928 (+ 583us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:21.115386 (+ 458us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:21.116479 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.117675 (+ 1196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.118635 (+ 960us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.119208 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.119418 (+ 210us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.123045 (+ 3627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:21.123051 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:21.123068 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:21.123112 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:21.125240 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:21.125841 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:21.125847 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:21.125852 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:21.125979 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:21.126301 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:21.126304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:21.126306 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:21.126828 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:21.127035 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:21.127358 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 22:12:21.127715 (+ 357us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":388,"thread_start_us":173,"threads_started":1}
I20260811 22:12:21.130918 13065 raft_consensus.cc:348] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } }
I20260811 22:12:21.131280 13065 raft_consensus.cc:374] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:21.131385 13065 raft_consensus.cc:729] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 805a0aed16e246bc8c7228ea66c2d766, State: Initialized, Role: FOLLOWER
I20260811 22:12:21.131901 13065 consensus_queue.cc:260] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } }
I20260811 22:12:21.132047 13065 raft_consensus.cc:388] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:21.132092 13065 raft_consensus.cc:482] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:21.132175 13065 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:21.133450 13065 raft_consensus.cc:504] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } }
I20260811 22:12:21.133893 13065 leader_election.cc:302] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 805a0aed16e246bc8c7228ea66c2d766; no voters:
I20260811 22:12:21.134219 13065 leader_election.cc:288] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:21.134419 13070 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:21.134781 13070 raft_consensus.cc:686] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 1 LEADER]: Becoming Leader. State: Replica: 805a0aed16e246bc8c7228ea66c2d766, State: Running, Role: LEADER
I20260811 22:12:21.135200 13070 consensus_queue.cc:237] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } }
I20260811 22:12:21.135846 13065 sys_catalog.cc:564] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:21.137446 13072 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "805a0aed16e246bc8c7228ea66c2d766" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } } }
I20260811 22:12:21.137444 13071 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 805a0aed16e246bc8c7228ea66c2d766. Latest consensus state: current_term: 1 leader_uuid: "805a0aed16e246bc8c7228ea66c2d766" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } } }
I20260811 22:12:21.137658 13071 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:21.137656 13072 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:21.138231 13084 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:21.142045 13084 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:21.149542 13084 catalog_manager.cc:1347] Generated new cluster ID: 9efa9586523c4dcda91493e4e1d6bc5c
I20260811 22:12:21.149670 13084 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:21.156425 13084 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:21.157634 13084 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:21.163671 13084 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Generated new TSK 0
I20260811 22:12:21.164873 13084 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:12:21.209890 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:36885
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:21.340014 13093 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:21.340385 13093 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:21.340472 13093 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:21.344972 13093 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:21.345118 13093 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:21.345248 13093 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:12:21.351285 13093 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:36885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13093
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:21.352559 13093 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:21.353855 13093 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:21.364797 13098 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:21.364867 13102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:21.364892 13099 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:21.365052 13093 server_base.cc:1034] running on GCE node
I20260811 22:12:21.365782 13093 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:21.366643 13093 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:21.367882 13093 hybrid_clock.cc:648] HybridClock initialized: now 1786486341367841 us; error 55 us; skew 500 ppm
Aug 11 22:12:21 dist-test-slave-vxj2 krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:21.371171 13093 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:12:21.372274 13093 webserver.cc:460] Webserver started at http://127.28.47.129:33219/ using document root <none> and password file <none>
I20260811 22:12:21.372985 13093 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:21.373099 13093 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:21.373401 13093 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:21.376080 13093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "385295a90d342b06ecaaff51d7f80953"
server_key_iv: "211b1affd6a0ca0012fa16869ac8098a"
server_key_version: "encryptionkey@0"
I20260811 22:12:21.376809 13093 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "385295a90d342b06ecaaff51d7f80953"
server_key_iv: "211b1affd6a0ca0012fa16869ac8098a"
server_key_version: "encryptionkey@0"
I20260811 22:12:21.382354 13093 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260811 22:12:21.387252 13110 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:21.389278 13093 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:12:21.389462 13093 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "385295a90d342b06ecaaff51d7f80953"
server_key_iv: "211b1affd6a0ca0012fa16869ac8098a"
server_key_version: "encryptionkey@0"
I20260811 22:12:21.389631 13093 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:21.404771 13093 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:21.408733 13093 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:21.408981 13093 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:21.409758 13093 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:21.410729 13093 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:21.410806 13093 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:21.410887 13093 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:21.410931 13093 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:21.421833 13093 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:42677
I20260811 22:12:21.421861 13245 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:42677 every 8 connection(s)
I20260811 22:12:21.422854 13093 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:21.429488 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 13093
I20260811 22:12:21.429628 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:12:21.429947 28862 external_mini_cluster.cc:1428] Setting key 1278bf83271e012cc680d57bfdd22379
Aug 11 22:12:21 dist-test-slave-vxj2 krb5kdc[12974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:21.438899 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.424951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:54017 (local address 127.28.47.190:36885)
0811 22:12:21.425124 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:21.425128 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:21.426205 (+ 1077us) server_negotiation.cc:407] Connection header received
0811 22:12:21.427639 (+ 1434us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:21.427643 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:21.427709 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:21.427794 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:21.429029 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.429888 (+ 859us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.431399 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.431723 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.431853 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.434766 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:21.434770 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:21.434772 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:21.434799 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:21.436483 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:21.437205 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:21.437211 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:21.437213 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:21.437305 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:21.437697 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:21.437702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:21.437704 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:21.437921 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:21.438032 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:21.438587 (+ 555us) server_negotiation.cc:299] Negotiation successful
0811 22:12:21.438730 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:12:21.439810 13249 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.425459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:54017)
0811 22:12:21.426038 (+ 579us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:21.426083 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:21.427418 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:21.427974 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:21.427981 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:21.428364 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:21.428871 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:21.428885 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.430100 (+ 1215us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.430109 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:21.431284 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:21.431296 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.431903 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.431906 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:21.432060 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.432833 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:21.432850 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:21.434557 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:21.436698 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:21.436704 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:21.436719 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:21.437060 (+ 341us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:21.437416 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:21.437420 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:21.437422 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:21.437582 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:21.438033 (+ 451us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:21.438041 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:21.438330 (+ 289us) client_negotiation.cc:769] Sending connection context
0811 22:12:21.438531 (+ 201us) client_negotiation.cc:240] Negotiation successful
0811 22:12:21.438909 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":135,"threads_started":1}
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
I20260811 22:12:21.441358 13246 heartbeater.cc:347] Connected to a master server at 127.28.47.190:36885
I20260811 22:12:21.441630 13246 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:21.442322 13246 heartbeater.cc:511] Master 127.28.47.190:36885 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:12:21.444201 13022 ts_manager.cc:194] Registered new tserver with Master: bc33f14b2a27475aa5f5dd7c49b63bd0 (127.28.47.129:42677)
I20260811 22:12:21.445562 13022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:54017
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:12:21.465857 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:36885
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:21.588846 13252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:21.589257 13252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:21.589370 13252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:21.593705 13252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:21.593783 13252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:21.593847 13252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:12:21.599725 13252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:36885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:21.600996 13252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:21.602188 13252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:21.613334 13257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:21.613353 13260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:21.613332 13258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:21.613566 13252 server_base.cc:1034] running on GCE node
I20260811 22:12:21.614259 13252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:21.615101 13252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:21.616302 13252 hybrid_clock.cc:648] HybridClock initialized: now 1786486341616276 us; error 37 us; skew 500 ppm
Aug 11 22:12:21 dist-test-slave-vxj2 krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:21.624248 13252 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:12:21.625516 13252 webserver.cc:460] Webserver started at http://127.28.47.130:40177/ using document root <none> and password file <none>
I20260811 22:12:21.626169 13252 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:21.626245 13252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:21.626510 13252 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:21.628975 13252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "f86c0d3f5f4c43eb80366236476c9c2b"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "c740603e4c4f9db221bbcad308903ca5"
server_key_iv: "dff4eb2d421bcc710b85db24fbc4a06b"
server_key_version: "encryptionkey@0"
I20260811 22:12:21.629702 13252 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "f86c0d3f5f4c43eb80366236476c9c2b"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "c740603e4c4f9db221bbcad308903ca5"
server_key_iv: "dff4eb2d421bcc710b85db24fbc4a06b"
server_key_version: "encryptionkey@0"
I20260811 22:12:21.633795 13252 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:12:21.636785 13267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:21.638556 13252 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:21.638731 13252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "f86c0d3f5f4c43eb80366236476c9c2b"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "c740603e4c4f9db221bbcad308903ca5"
server_key_iv: "dff4eb2d421bcc710b85db24fbc4a06b"
server_key_version: "encryptionkey@0"
I20260811 22:12:21.638885 13252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:21.660030 13252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:21.663757 13252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:21.663987 13252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:21.664688 13252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:21.665680 13252 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:21.665766 13252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:21.665856 13252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:21.665907 13252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:21.676609 13252 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:42583
I20260811 22:12:21.676635 13412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:42583 every 8 connection(s)
I20260811 22:12:21.677680 13252 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:21.684810 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 13252
I20260811 22:12:21.684930 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:12:21.685318 28862 external_mini_cluster.cc:1428] Setting key ed6a4a146665b7980b91e0f922ba168f
Aug 11 22:12:21 dist-test-slave-vxj2 krb5kdc[12974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:21.693727 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.679758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:44815 (local address 127.28.47.190:36885)
0811 22:12:21.679947 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:21.679953 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:21.680941 (+ 988us) server_negotiation.cc:407] Connection header received
0811 22:12:21.682387 (+ 1446us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:21.682391 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:21.682443 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:21.682533 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:21.683910 (+ 1377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.684450 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.685972 (+ 1522us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.686272 (+ 300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.686389 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.689698 (+ 3309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:21.689701 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:21.689703 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:21.689732 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:21.691256 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:21.691897 (+ 641us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:21.691900 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:21.691901 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:21.691986 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:21.692377 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:21.692380 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:21.692382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:21.692593 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:21.692708 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:21.693309 (+ 601us) server_negotiation.cc:299] Negotiation successful
0811 22:12:21.693451 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260811 22:12:21.694780 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.680234 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:44815)
0811 22:12:21.680766 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:21.680812 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:21.682138 (+ 1326us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:21.682760 (+ 622us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:21.682767 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:21.683203 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:21.683735 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:21.683751 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.684625 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.684628 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:21.685825 (+ 1197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:21.685840 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.686458 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.686461 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:21.686677 (+ 216us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.687372 (+ 695us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:21.687395 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:21.689465 (+ 2070us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:21.691419 (+ 1954us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:21.691426 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:21.691445 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:21.691765 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:21.692087 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:21.692091 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:21.692093 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:21.692269 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:21.692708 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:21.692717 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:21.693014 (+ 297us) client_negotiation.cc:769] Sending connection context
0811 22:12:21.693248 (+ 234us) client_negotiation.cc:240] Negotiation successful
0811 22:12:21.693799 (+ 551us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":130,"threads_started":1}
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:12:21.696241 13413 heartbeater.cc:347] Connected to a master server at 127.28.47.190:36885
I20260811 22:12:21.696547 13413 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:21.697227 13413 heartbeater.cc:511] Master 127.28.47.190:36885 requested a full tablet report, sending...
I20260811 22:12:21.698437 13022 ts_manager.cc:194] Registered new tserver with Master: f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
I20260811 22:12:21.699270 13022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:44815
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:12:21.716990 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:36885
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:21.840940 13419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:21.841315 13419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:21.841390 13419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:21.846077 13419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:21.846146 13419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:21.846239 13419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:12:21.852056 13419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:36885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:21.853380 13419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:21.854645 13419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:21.865125 13429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:21.865136 13425 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:21.865121 13426 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:21.866547 13419 server_base.cc:1034] running on GCE node
I20260811 22:12:21.867105 13419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:21.867831 13419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:21.869097 13419 hybrid_clock.cc:648] HybridClock initialized: now 1786486341869076 us; error 31 us; skew 500 ppm
Aug 11 22:12:21 dist-test-slave-vxj2 krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:21.872200 13419 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:12:21.873360 13419 webserver.cc:460] Webserver started at http://127.28.47.131:45843/ using document root <none> and password file <none>
I20260811 22:12:21.874039 13419 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:21.874135 13419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:21.874359 13419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:21.876639 13419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "460ccdb7881b4417a9454af680232eca"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "71909dca7e64f0f9faa31c5c19e49f22"
server_key_iv: "a96bc17e97c5ff24d9dbad94d44d96fd"
server_key_version: "encryptionkey@0"
I20260811 22:12:21.877308 13419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "460ccdb7881b4417a9454af680232eca"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "71909dca7e64f0f9faa31c5c19e49f22"
server_key_iv: "a96bc17e97c5ff24d9dbad94d44d96fd"
server_key_version: "encryptionkey@0"
I20260811 22:12:21.881405 13419 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:12:21.883994 13438 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:21.885438 13419 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:21.885593 13419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "460ccdb7881b4417a9454af680232eca"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "71909dca7e64f0f9faa31c5c19e49f22"
server_key_iv: "a96bc17e97c5ff24d9dbad94d44d96fd"
server_key_version: "encryptionkey@0"
I20260811 22:12:21.885720 13419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:21.899652 13419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:21.903548 13419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:21.903793 13419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:21.904476 13419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:21.905567 13419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:21.905640 13419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:21.905712 13419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:21.905766 13419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:21.915902 13419 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:37001
I20260811 22:12:21.915956 13573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:37001 every 8 connection(s)
I20260811 22:12:21.916868 13419 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:21.926049 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 13419
I20260811 22:12:21.926203 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:12:21.926535 28862 external_mini_cluster.cc:1428] Setting key 5bbab7e0544edad3d089367633ceb508
Aug 11 22:12:21 dist-test-slave-vxj2 krb5kdc[12974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486341, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:21.931697 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.918865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:43221 (local address 127.28.47.190:36885)
0811 22:12:21.919024 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:21.919028 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:21.919911 (+ 883us) server_negotiation.cc:407] Connection header received
0811 22:12:21.921436 (+ 1525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:21.921440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:21.921506 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:21.921613 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:21.922886 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.923424 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.924642 (+ 1218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.924886 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.925007 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.928031 (+ 3024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:21.928035 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:21.928038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:21.928069 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:21.929700 (+ 1631us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:21.930254 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:21.930256 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:21.930258 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:21.930333 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:21.930666 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:21.930668 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:21.930669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:21.930856 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:21.930940 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:21.931463 (+ 523us) server_negotiation.cc:299] Negotiation successful
0811 22:12:21.931572 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:12:21.932657 13576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.919210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:43221)
0811 22:12:21.919725 (+ 515us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:21.919769 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:21.921115 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:21.921809 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:21.921816 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:21.922197 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:21.922710 (+ 513us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:21.922724 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.923559 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.923565 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:21.924502 (+ 937us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:21.924513 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.925056 (+ 543us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.925060 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:21.925266 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.925963 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:21.925980 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:21.927804 (+ 1824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:21.929847 (+ 2043us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:21.929852 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:21.929865 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:21.930157 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:21.930433 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:21.930435 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:21.930437 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:21.930579 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:21.930951 (+ 372us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:21.930957 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:21.931212 (+ 255us) client_negotiation.cc:769] Sending connection context
0811 22:12:21.931401 (+ 189us) client_negotiation.cc:240] Negotiation successful
0811 22:12:21.931678 (+ 277us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315,"thread_start_us":147,"threads_started":1}
I20260811 22:12:21.934195 13574 heartbeater.cc:347] Connected to a master server at 127.28.47.190:36885
I20260811 22:12:21.934475 13574 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:21.935045 13574 heartbeater.cc:511] Master 127.28.47.190:36885 requested a full tablet report, sending...
I20260811 22:12:21.936304 13022 ts_manager.cc:194] Registered new tserver with Master: 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001)
I20260811 22:12:21.937044 13022 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:43221
I20260811 22:12:21.941042 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:12:21.951611 13068 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.943273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57508 (local address 127.28.47.190:36885)
0811 22:12:21.943465 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:21.943470 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:21.943767 (+ 297us) server_negotiation.cc:407] Connection header received
0811 22:12:21.943906 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:21.943909 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:21.943955 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:21.944027 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:21.944769 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.945379 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.946251 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.946485 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.946581 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.948194 (+ 1613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:21.948197 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:21.948199 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:21.948228 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:21.949947 (+ 1719us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:21.950390 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:21.950393 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:21.950394 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:21.950471 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:21.950705 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:21.950707 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:21.950708 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:21.950984 (+ 276us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:21.951091 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:21.951305 (+ 214us) server_negotiation.cc:299] Negotiation successful
0811 22:12:21.951411 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92}
I20260811 22:12:21.955284 13022 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57508:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:12:21.958683 13022 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:12:21.976119 13589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.969735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:42583 (local address 127.0.0.1:50936)
0811 22:12:21.970249 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:21.970266 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:21.970383 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:21.970952 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:21.970958 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:21.970984 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:21.971205 (+ 221us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:21.971215 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.972743 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.972746 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:21.974323 (+ 1577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:21.974335 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.975499 (+ 1164us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.975502 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:21.975642 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.975677 (+ 35us) client_negotiation.cc:769] Sending connection context
0811 22:12:21.975812 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 22:12:21.975956 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":413,"mutex_wait_us":133,"thread_start_us":108,"threads_started":1}
I20260811 22:12:21.976323 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.969133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40360 (local address 127.28.47.129:42677)
0811 22:12:21.969676 (+ 543us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:21.969684 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:21.970123 (+ 439us) server_negotiation.cc:407] Connection header received
0811 22:12:21.970469 (+ 346us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:21.970479 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:21.970716 (+ 237us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:21.970852 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:21.971363 (+ 511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.972591 (+ 1228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.974683 (+ 2092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.975476 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.975818 (+ 342us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.975980 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:21.976036 (+ 56us) server_negotiation.cc:299] Negotiation successful
0811 22:12:21.976152 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":129,"threads_started":1}
I20260811 22:12:21.976593 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.969427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50936 (local address 127.28.47.130:42583)
0811 22:12:21.970156 (+ 729us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:21.970169 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:21.970527 (+ 358us) server_negotiation.cc:407] Connection header received
0811 22:12:21.970623 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:21.970627 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:21.970780 (+ 153us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:21.970956 (+ 176us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:21.971366 (+ 410us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.972556 (+ 1190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.974459 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.975333 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.975513 (+ 180us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.975694 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:21.976356 (+ 662us) server_negotiation.cc:299] Negotiation successful
0811 22:12:21.976466 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":614,"thread_start_us":147,"threads_started":1}
I20260811 22:12:21.977002 13585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.968973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:42677 (local address 127.0.0.1:40360)
0811 22:12:21.969967 (+ 994us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:21.970028 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:21.970251 (+ 223us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:21.971047 (+ 796us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:21.971050 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:21.971068 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:21.971207 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:21.971214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.972741 (+ 1527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.972745 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:21.974552 (+ 1807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:21.974557 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.975815 (+ 1258us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.975820 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:21.975911 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.975934 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:12:21.976003 (+ 69us) client_negotiation.cc:240] Negotiation successful
0811 22:12:21.976077 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":802,"mutex_wait_us":381,"thread_start_us":120,"threads_started":1}
I20260811 22:12:21.978158 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.969841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:37001 (local address 127.0.0.1:46768)
0811 22:12:21.970350 (+ 509us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:21.970363 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:21.970449 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:21.971219 (+ 770us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:21.971222 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:21.971243 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:21.971365 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:21.971373 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.972799 (+ 1426us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.972805 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:21.974542 (+ 1737us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:21.974555 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.977731 (+ 3176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.977738 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:21.977838 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.977864 (+ 26us) client_negotiation.cc:769] Sending connection context
0811 22:12:21.977950 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:12:21.978035 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":444,"mutex_wait_us":333,"thread_start_us":113,"threads_started":1}
I20260811 22:12:21.978268 13590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:21.970321 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46768 (local address 127.28.47.131:37001)
0811 22:12:21.970788 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:21.970801 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:21.970822 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:12:21.970896 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:21.970900 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:21.971064 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:21.971213 (+ 149us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:21.971497 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.972644 (+ 1147us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.976642 (+ 3998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:21.977596 (+ 954us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:21.977712 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:21.977875 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:21.978031 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 22:12:21.978130 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":149,"threads_started":1}
I20260811 22:12:21.979943 13170 tablet_service.cc:1467] Processing CreateTablet for tablet d0fcb0c2b01c4ba193f3e34a2e564168 (DEFAULT_TABLE table=test-table [id=08e29f35cb624e08a4c935ca21ea4708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:21.979946 13330 tablet_service.cc:1467] Processing CreateTablet for tablet d0fcb0c2b01c4ba193f3e34a2e564168 (DEFAULT_TABLE table=test-table [id=08e29f35cb624e08a4c935ca21ea4708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:21.981062 13330 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0fcb0c2b01c4ba193f3e34a2e564168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:21.981086 13170 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0fcb0c2b01c4ba193f3e34a2e564168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:21.981307 13498 tablet_service.cc:1467] Processing CreateTablet for tablet d0fcb0c2b01c4ba193f3e34a2e564168 (DEFAULT_TABLE table=test-table [id=08e29f35cb624e08a4c935ca21ea4708]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:21.982376 13498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d0fcb0c2b01c4ba193f3e34a2e564168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:21.987821 13591 tablet_bootstrap.cc:492] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b: Bootstrap starting.
I20260811 22:12:21.989552 13592 tablet_bootstrap.cc:492] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Bootstrap starting.
I20260811 22:12:21.989810 13594 tablet_bootstrap.cc:492] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca: Bootstrap starting.
I20260811 22:12:21.991882 13591 tablet_bootstrap.cc:654] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:21.991972 13592 tablet_bootstrap.cc:654] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:21.992573 13594 tablet_bootstrap.cc:654] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:21.992887 13592 log.cc:824] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:21.992938 13591 log.cc:824] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:21.993570 13594 log.cc:824] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:21.995255 13592 tablet_bootstrap.cc:492] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: No bootstrap required, opened a new log
I20260811 22:12:21.995255 13591 tablet_bootstrap.cc:492] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b: No bootstrap required, opened a new log
I20260811 22:12:21.995482 13594 tablet_bootstrap.cc:492] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca: No bootstrap required, opened a new log
I20260811 22:12:21.995488 13591 ts_tablet_manager.cc:1397] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 22:12:21.995488 13592 ts_tablet_manager.cc:1397] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:12:21.995698 13594 ts_tablet_manager.cc:1397] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:12:21.999231 13594 raft_consensus.cc:348] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } }
I20260811 22:12:21.999549 13594 raft_consensus.cc:374] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:21.999637 13594 raft_consensus.cc:729] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460ccdb7881b4417a9454af680232eca, State: Initialized, Role: FOLLOWER
I20260811 22:12:21.999758 13591 raft_consensus.cc:348] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } }
I20260811 22:12:22.000034 13591 raft_consensus.cc:374] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:22.000141 13591 raft_consensus.cc:729] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f86c0d3f5f4c43eb80366236476c9c2b, State: Initialized, Role: FOLLOWER
I20260811 22:12:22.000190 13594 consensus_queue.cc:260] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } }
I20260811 22:12:22.000643 13591 consensus_queue.cc:260] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } }
I20260811 22:12:22.000658 13592 raft_consensus.cc:348] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } }
I20260811 22:12:22.000950 13592 raft_consensus.cc:374] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:22.001019 13592 raft_consensus.cc:729] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc33f14b2a27475aa5f5dd7c49b63bd0, State: Initialized, Role: FOLLOWER
I20260811 22:12:22.001042 13574 heartbeater.cc:503] Master 127.28.47.190:36885 was elected leader, sending a full tablet report...
I20260811 22:12:22.001482 13594 ts_tablet_manager.cc:1428] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:12:22.001482 13592 consensus_queue.cc:260] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } }
I20260811 22:12:22.002159 13246 heartbeater.cc:503] Master 127.28.47.190:36885 was elected leader, sending a full tablet report...
I20260811 22:12:22.002563 13592 ts_tablet_manager.cc:1428] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:12:22.003176 13413 heartbeater.cc:503] Master 127.28.47.190:36885 was elected leader, sending a full tablet report...
I20260811 22:12:22.003213 13591 ts_tablet_manager.cc:1428] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260811 22:12:22.167997 13575 tablet.cc:2369] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:12:22.174779 13247 tablet.cc:2369] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:12:22.179036 13414 tablet.cc:2369] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:12:22.256749 13602 raft_consensus.cc:482] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:22.257021 13602 raft_consensus.cc:504] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } }
I20260811 22:12:22.258471 13602 leader_election.cc:288] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers bc33f14b2a27475aa5f5dd7c49b63bd0 (127.28.47.129:42677), 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001)
I20260811 22:12:22.263549 13590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:22.259405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:50015 (local address 127.28.47.131:37001)
0811 22:12:22.259557 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:22.259562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:22.259605 (+ 43us) server_negotiation.cc:407] Connection header received
0811 22:12:22.259840 (+ 235us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:22.259855 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:22.259905 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:22.260004 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:22.260513 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.261200 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.262446 (+ 1246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.262901 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.263003 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:22.263060 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:22.263318 (+ 258us) server_negotiation.cc:299] Negotiation successful
0811 22:12:22.263415 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:12:22.263566 13605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:22.259228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:37001 (local address 127.28.47.130:50015)
0811 22:12:22.259514 (+ 286us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:22.259531 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:22.259637 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:22.260049 (+ 412us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:22.260051 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:22.260070 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:22.260361 (+ 291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:22.260369 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.261335 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.261338 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:22.262310 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:22.262323 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.263053 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.263056 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:22.263176 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:22.263205 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:12:22.263284 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:12:22.263371 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":3,"thread_start_us":127,"threads_started":1}
I20260811 22:12:22.263654 13604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:22.258744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:42677 (local address 127.28.47.130:44635)
0811 22:12:22.259152 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:22.259170 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:22.259408 (+ 238us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:22.259834 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:22.259837 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:22.259861 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:22.260024 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:22.260032 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.261177 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.261181 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:22.262325 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:22.262336 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.263195 (+ 859us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.263197 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:22.263321 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:22.263346 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:12:22.263424 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:12:22.263492 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"mutex_wait_us":22,"thread_start_us":138,"threads_started":1}
I20260811 22:12:22.263681 13586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:22.258946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:44635 (local address 127.28.47.129:42677)
0811 22:12:22.259147 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:22.259152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:22.259304 (+ 152us) server_negotiation.cc:407] Connection header received
0811 22:12:22.259589 (+ 285us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:22.259595 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:22.259653 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:22.259734 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:22.260259 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.261016 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.262537 (+ 1521us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.263061 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.263190 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:22.263244 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:22.263468 (+ 224us) server_negotiation.cc:299] Negotiation successful
0811 22:12:22.263557 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:12:22.264369 13191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0fcb0c2b01c4ba193f3e34a2e564168" candidate_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" is_pre_election: true
I20260811 22:12:22.264367 13519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0fcb0c2b01c4ba193f3e34a2e564168" candidate_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460ccdb7881b4417a9454af680232eca" is_pre_election: true
I20260811 22:12:22.264703 13519 raft_consensus.cc:2454] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f86c0d3f5f4c43eb80366236476c9c2b in term 0.
I20260811 22:12:22.264712 13191 raft_consensus.cc:2454] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f86c0d3f5f4c43eb80366236476c9c2b in term 0.
I20260811 22:12:22.265250 13277 leader_election.cc:302] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc33f14b2a27475aa5f5dd7c49b63bd0, f86c0d3f5f4c43eb80366236476c9c2b; no voters:
I20260811 22:12:22.265591 13602 raft_consensus.cc:2781] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:22.265698 13602 raft_consensus.cc:482] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:22.265745 13602 raft_consensus.cc:3037] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:22.266899 13602 raft_consensus.cc:504] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } }
I20260811 22:12:22.267289 13602 leader_election.cc:288] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [CANDIDATE]: Term 1 election: Requested vote from peers bc33f14b2a27475aa5f5dd7c49b63bd0 (127.28.47.129:42677), 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001)
I20260811 22:12:22.267766 13191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0fcb0c2b01c4ba193f3e34a2e564168" candidate_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0"
I20260811 22:12:22.267778 13519 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d0fcb0c2b01c4ba193f3e34a2e564168" candidate_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460ccdb7881b4417a9454af680232eca"
I20260811 22:12:22.267928 13519 raft_consensus.cc:3037] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:22.267947 13191 raft_consensus.cc:3037] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:22.269059 13519 raft_consensus.cc:2454] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f86c0d3f5f4c43eb80366236476c9c2b in term 1.
I20260811 22:12:22.269332 13191 raft_consensus.cc:2454] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f86c0d3f5f4c43eb80366236476c9c2b in term 1.
I20260811 22:12:22.269469 13277 leader_election.cc:302] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 460ccdb7881b4417a9454af680232eca, f86c0d3f5f4c43eb80366236476c9c2b; no voters:
I20260811 22:12:22.269696 13602 raft_consensus.cc:2781] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:22.269948 13602 raft_consensus.cc:686] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [term 1 LEADER]: Becoming Leader. State: Replica: f86c0d3f5f4c43eb80366236476c9c2b, State: Running, Role: LEADER
I20260811 22:12:22.270416 13602 consensus_queue.cc:237] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } }
I20260811 22:12:22.274402 13022 catalog_manager.cc:5495] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b reported cstate change: term changed from 0 to 1, leader changed from <none> to f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130). New cstate: current_term: 1 leader_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } health_report { overall_health: HEALTHY } } }
I20260811 22:12:22.343691 13587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:22.339147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50940 (local address 127.28.47.130:42583)
0811 22:12:22.339285 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:22.339289 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:22.339304 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:12:22.339393 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:22.339396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:22.339457 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:22.339573 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:22.339967 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.340588 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.341728 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.342010 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.342130 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:22.342536 (+ 406us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:22.343166 (+ 630us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:22.343268 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:22.343406 (+ 138us) server_negotiation.cc:299] Negotiation successful
0811 22:12:22.343502 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:12:22.350962 13519 raft_consensus.cc:1261] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca [term 1 FOLLOWER]: Refusing update from remote peer f86c0d3f5f4c43eb80366236476c9c2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:22.351030 13191 raft_consensus.cc:1261] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Refusing update from remote peer f86c0d3f5f4c43eb80366236476c9c2b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:22.351800 13602 consensus_queue.cc:1035] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:22.352048 13606 consensus_queue.cc:1035] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [LEADER]: Connected to new peer: Peer: permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:22.359057 13609 mvcc.cc:205] Tried to move back new op lower bound from 7317448058260336640 to 7317448057942532096. Current Snapshot: MvccSnapshot[applied={T|T < 7317448058260336640}]
I20260811 22:12:22.362250 13611 mvcc.cc:205] Tried to move back new op lower bound from 7317448058260336640 to 7317448057942532096. Current Snapshot: MvccSnapshot[applied={T|T < 7317448058260336640}]
I20260811 22:12:22.370419 13022 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:22.370594 13022 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:57508:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:22.373338 13022 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Sending DeleteTablet for 3 replicas of tablet d0fcb0c2b01c4ba193f3e34a2e564168
I20260811 22:12:22.374445 13330 tablet_service.cc:1514] Processing DeleteTablet for tablet d0fcb0c2b01c4ba193f3e34a2e564168 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:22 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:50936
I20260811 22:12:22.374568 13170 tablet_service.cc:1514] Processing DeleteTablet for tablet d0fcb0c2b01c4ba193f3e34a2e564168 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:22 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:40360
I20260811 22:12:22.374594 13498 tablet_service.cc:1514] Processing DeleteTablet for tablet d0fcb0c2b01c4ba193f3e34a2e564168 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:22 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:46768
I20260811 22:12:22.375124 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 13093
I20260811 22:12:22.375156 13622 tablet_replica.cc:330] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca: stopping tablet replica
I20260811 22:12:22.382915 13622 raft_consensus.cc:2229] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:22.383538 13622 raft_consensus.cc:2258] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:22.384168 13621 tablet_replica.cc:330] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b: stopping tablet replica
I20260811 22:12:22.384636 13621 raft_consensus.cc:2229] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:12:22.385208 13621 raft_consensus.cc:2258] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:22.386708 13622 ts_tablet_manager.cc:1905] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:22.387966 13621 ts_tablet_manager.cc:1905] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:22.389477 13622 ts_tablet_manager.cc:1918] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:22.389632 13622 log.cc:1196] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/d0fcb0c2b01c4ba193f3e34a2e564168
I20260811 22:12:22.390040 13622 ts_tablet_manager.cc:1939] T d0fcb0c2b01c4ba193f3e34a2e564168 P 460ccdb7881b4417a9454af680232eca: Deleting consensus metadata
I20260811 22:12:22.390967 13621 ts_tablet_manager.cc:1918] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:22.391085 13621 log.cc:1196] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/d0fcb0c2b01c4ba193f3e34a2e564168
I20260811 22:12:22.391487 13621 ts_tablet_manager.cc:1939] T d0fcb0c2b01c4ba193f3e34a2e564168 P f86c0d3f5f4c43eb80366236476c9c2b: Deleting consensus metadata
I20260811 22:12:22.391572 13008 catalog_manager.cc:4841] TS 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001): tablet d0fcb0c2b01c4ba193f3e34a2e564168 (table test-table [id=08e29f35cb624e08a4c935ca21ea4708]) successfully deleted
I20260811 22:12:22.393031 13004 catalog_manager.cc:4841] TS f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583): tablet d0fcb0c2b01c4ba193f3e34a2e564168 (table test-table [id=08e29f35cb624e08a4c935ca21ea4708]) successfully deleted
W20260811 22:12:22.395694 13008 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:42677 (error 108)
I20260811 22:12:22.396766 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 13252
I20260811 22:12:22.397050 13588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:22.396660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:42677 (local address 127.0.0.1:40376)
0811 22:12:22.396837 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:22.396936 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:42677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 22:12:22.399919 13008 catalog_manager.cc:4584] TS bc33f14b2a27475aa5f5dd7c49b63bd0 (127.28.47.129:42677): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d0fcb0c2b01c4ba193f3e34a2e564168: Network error: Client connection negotiation failed: client connection to 127.28.47.129:42677: connect: Connection refused (error 111)
I20260811 22:12:22.408295 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 13419
I20260811 22:12:22.420203 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 12988
I20260811 22:12:22.431234 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:36885
--webserver_interface=127.28.47.190
--webserver_port=46509
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:36885
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:22.551610 13624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:22.552002 13624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:22.552088 13624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:22.556501 13624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:22.556649 13624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:22.556689 13624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:22.556727 13624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:22.556766 13624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:22.563143 13624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:36885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:36885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=46509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:22.564471 13624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:22.565630 13624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:22.580340 13634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:22.580513 13624 server_base.cc:1034] running on GCE node
W20260811 22:12:22.580344 13637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:22.580708 13633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:22.581573 13624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:22.582850 13624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:22.584067 13624 hybrid_clock.cc:648] HybridClock initialized: now 1786486342584015 us; error 69 us; skew 500 ppm
Aug 11 22:12:22 dist-test-slave-vxj2 krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486342, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:22.587271 13624 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:12:22.588363 13624 webserver.cc:460] Webserver started at http://127.28.47.190:46509/ using document root <none> and password file <none>
I20260811 22:12:22.589028 13624 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:22.589190 13624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:22.593595 13624 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 22:12:22.596124 13644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:22.597770 13624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:22.597968 13624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "805a0aed16e246bc8c7228ea66c2d766"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "7235c6e74a54b7c367e4e87a2759d12d"
server_key_iv: "b31411d801e9ee364fd8f478cbaedca1"
server_key_version: "encryptionkey@0"
I20260811 22:12:22.598656 13624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:22.614066 13624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:22.618000 13624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:22.618256 13624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:22.626376 13624 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:36885
I20260811 22:12:22.626722 13710 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:36885 every 8 connection(s)
I20260811 22:12:22.627668 13624 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:22.629576 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 13624
I20260811 22:12:22.630136 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:42677
--local_ip_for_outbound_sockets=127.28.47.129
--tserver_master_addrs=127.28.47.190:36885
--webserver_port=33219
--webserver_interface=127.28.47.129
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:12:22.633811 13711 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:12:22.635869 13711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Bootstrap starting.
I20260811 22:12:22.656858 13711 log.cc:824] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:22.667104 13711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:12:22.667599 13711 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Bootstrap complete.
I20260811 22:12:22.672425 13711 raft_consensus.cc:348] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } }
I20260811 22:12:22.672958 13711 raft_consensus.cc:729] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 805a0aed16e246bc8c7228ea66c2d766, State: Initialized, Role: FOLLOWER
I20260811 22:12:22.673627 13711 consensus_queue.cc:260] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } }
I20260811 22:12:22.673834 13711 raft_consensus.cc:388] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:22.673942 13711 raft_consensus.cc:482] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:22.674088 13711 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:12:22.676564 13711 raft_consensus.cc:504] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } }
I20260811 22:12:22.676983 13711 leader_election.cc:302] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 805a0aed16e246bc8c7228ea66c2d766; no voters:
I20260811 22:12:22.677423 13711 leader_election.cc:288] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:12:22.677531 13715 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:12:22.677754 13715 raft_consensus.cc:686] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 2 LEADER]: Becoming Leader. State: Replica: 805a0aed16e246bc8c7228ea66c2d766, State: Running, Role: LEADER
I20260811 22:12:22.678177 13715 consensus_queue.cc:237] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } }
I20260811 22:12:22.678969 13711 sys_catalog.cc:564] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:22.679898 13717 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "805a0aed16e246bc8c7228ea66c2d766" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } } }
I20260811 22:12:22.679952 13718 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 805a0aed16e246bc8c7228ea66c2d766. Latest consensus state: current_term: 2 leader_uuid: "805a0aed16e246bc8c7228ea66c2d766" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } } }
I20260811 22:12:22.680047 13717 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:22.680083 13718 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:22.681423 13725 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:22.686313 13725 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:22.686990 13725 catalog_manager.cc:1259] Loaded cluster ID: 9efa9586523c4dcda91493e4e1d6bc5c
I20260811 22:12:22.687077 13725 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:22.688504 13725 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260811 22:12:22.691699 13725 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260811 22:12:22.697914 13725 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:22.698820 13725 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Loaded TSK: 0
I20260811 22:12:22.699982 13725 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:12:22.769308 13713 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:22.769625 13713 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:22.769726 13713 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:22.774034 13713 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:22.774123 13713 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:22.774235 13713 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:12:22.780349 13713 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:42677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=33219
--tserver_master_addrs=127.28.47.190:36885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13713
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:22.781803 13713 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:22.783016 13713 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:22.793462 13742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:22.793695 13741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:22.793984 13744 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:22.794322 13713 server_base.cc:1034] running on GCE node
I20260811 22:12:22.794819 13713 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:22.795598 13713 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:22.796792 13713 hybrid_clock.cc:648] HybridClock initialized: now 1786486342796773 us; error 32 us; skew 500 ppm
Aug 11 22:12:22 dist-test-slave-vxj2 krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486342, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:22.799979 13713 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:12:22.801093 13713 webserver.cc:460] Webserver started at http://127.28.47.129:33219/ using document root <none> and password file <none>
I20260811 22:12:22.801754 13713 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:22.801887 13713 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:22.806398 13713 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:12:22.809464 13753 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:22.811102 13713 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:12:22.811247 13713 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "385295a90d342b06ecaaff51d7f80953"
server_key_iv: "211b1affd6a0ca0012fa16869ac8098a"
server_key_version: "encryptionkey@0"
I20260811 22:12:22.811897 13713 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:22.831516 13713 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:22.834949 13713 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:22.835192 13713 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:22.835918 13713 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:22.837122 13761 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:12:22.842053 13713 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:12:22.842134 13713 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260811 22:12:22.842221 13713 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:12:22.844086 13713 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:12:22.844156 13713 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260811 22:12:22.844450 13761 tablet_bootstrap.cc:492] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Bootstrap starting.
I20260811 22:12:22.856233 13713 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:42677
I20260811 22:12:22.856612 13887 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:42677 every 8 connection(s)
I20260811 22:12:22.857775 13713 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:22.859253 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 13713
I20260811 22:12:22.859851 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:42583
--local_ip_for_outbound_sockets=127.28.47.130
--tserver_master_addrs=127.28.47.190:36885
--webserver_port=40177
--webserver_interface=127.28.47.130
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:12:22.867940 13761 log.cc:824] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:12:22 dist-test-slave-vxj2 krb5kdc[12974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486342, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:22.877434 13891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:22.860582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:45673)
0811 22:12:22.861589 (+ 1007us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:22.861654 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:22.862639 (+ 985us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:22.863910 (+ 1271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:22.863919 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:22.864428 (+ 509us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:22.864945 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:22.864962 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.866599 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.866602 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:22.867589 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:22.867601 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.868697 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.868702 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:22.868928 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:22.869712 (+ 784us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:22.869762 (+ 50us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:22.871540 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:22.873996 (+ 2456us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:22.874003 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:22.874017 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:22.874310 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:22.874711 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:22.874714 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:22.874716 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:22.874879 (+ 163us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:22.875516 (+ 637us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:22.875523 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:22.875792 (+ 269us) client_negotiation.cc:769] Sending connection context
0811 22:12:22.876001 (+ 209us) client_negotiation.cc:240] Negotiation successful
0811 22:12:22.876397 (+ 396us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768,"thread_start_us":168,"threads_started":1}
I20260811 22:12:22.877419 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:22.860622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:45673 (local address 127.28.47.190:36885)
0811 22:12:22.861246 (+ 624us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:22.861269 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:22.861827 (+ 558us) server_negotiation.cc:407] Connection header received
0811 22:12:22.863113 (+ 1286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:22.863124 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:22.863457 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:22.863832 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:22.865220 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.866365 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.867749 (+ 1384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.868508 (+ 759us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.868728 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:22.871785 (+ 3057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:22.871791 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:22.871806 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:22.871841 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:22.873841 (+ 2000us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:22.874459 (+ 618us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:22.874464 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:22.874468 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:22.874599 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:22.874986 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:22.874990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:22.874992 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:22.875389 (+ 397us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:22.875542 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:22.876122 (+ 580us) server_negotiation.cc:299] Negotiation successful
0811 22:12:22.876409 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":152,"threads_started":1}
I20260811 22:12:22.880116 13761 tablet_bootstrap.cc:492] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:12:22.880483 13888 heartbeater.cc:347] Connected to a master server at 127.28.47.190:36885
I20260811 22:12:22.880748 13888 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:22.880754 13761 tablet_bootstrap.cc:492] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Bootstrap complete.
I20260811 22:12:22.881505 13888 heartbeater.cc:511] Master 127.28.47.190:36885 requested a full tablet report, sending...
I20260811 22:12:22.882725 13761 ts_tablet_manager.cc:1397] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Time spent bootstrapping tablet: real 0.039s user 0.026s sys 0.010s
I20260811 22:12:22.884462 13661 ts_manager.cc:194] Registered new tserver with Master: bc33f14b2a27475aa5f5dd7c49b63bd0 (127.28.47.129:42677)
I20260811 22:12:22.886170 13761 raft_consensus.cc:348] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } }
I20260811 22:12:22.886824 13761 raft_consensus.cc:729] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc33f14b2a27475aa5f5dd7c49b63bd0, State: Initialized, Role: FOLLOWER
I20260811 22:12:22.887406 13761 consensus_queue.cc:260] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } }
I20260811 22:12:22.888509 13661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:45673
I20260811 22:12:22.888649 13761 ts_tablet_manager.cc:1428] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 22:12:22.891242 13888 heartbeater.cc:503] Master 127.28.47.190:36885 was elected leader, sending a full tablet report...
I20260811 22:12:22.897857 13895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:22.887802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:42677 (local address 127.0.0.1:40392)
0811 22:12:22.889636 (+ 1834us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:22.889673 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:22.891739 (+ 2066us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:22.892436 (+ 697us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:22.892446 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:22.892477 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:22.892638 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:22.892649 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.894330 (+ 1681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.894333 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:22.895695 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:22.895709 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.897005 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.897009 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:22.897133 (+ 124us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:22.897467 (+ 334us) client_negotiation.cc:769] Sending connection context
0811 22:12:22.897582 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 22:12:22.897712 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1651,"thread_start_us":116,"threads_started":1}
I20260811 22:12:22.897939 13896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:22.888627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40392 (local address 127.28.47.129:42677)
0811 22:12:22.889224 (+ 597us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:22.889238 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:22.889768 (+ 530us) server_negotiation.cc:407] Connection header received
0811 22:12:22.891967 (+ 2199us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:22.891974 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:22.892222 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:22.892414 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:22.892803 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.894184 (+ 1381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.895866 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:22.896763 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:22.896918 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:22.897096 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:22.897652 (+ 556us) server_negotiation.cc:299] Negotiation successful
0811 22:12:22.897779 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":450,"thread_start_us":113,"threads_started":1}
I20260811 22:12:22.899398 13811 tablet_service.cc:1514] Processing DeleteTablet for tablet d0fcb0c2b01c4ba193f3e34a2e564168 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:22 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:40392
I20260811 22:12:22.900677 13897 tablet_replica.cc:330] stopping tablet replica
I20260811 22:12:22.901101 13897 raft_consensus.cc:2229] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:22.901424 13897 raft_consensus.cc:2258] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:22.902664 13897 ts_tablet_manager.cc:1905] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:22.906972 13897 ts_tablet_manager.cc:1918] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:22.907125 13897 log.cc:1196] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal/wals/d0fcb0c2b01c4ba193f3e34a2e564168
I20260811 22:12:22.907424 13897 ts_tablet_manager.cc:1939] T d0fcb0c2b01c4ba193f3e34a2e564168 P bc33f14b2a27475aa5f5dd7c49b63bd0: Deleting consensus metadata
I20260811 22:12:22.909387 13649 catalog_manager.cc:4841] TS bc33f14b2a27475aa5f5dd7c49b63bd0 (127.28.47.129:42677): tablet d0fcb0c2b01c4ba193f3e34a2e564168 (table test-table [id=08e29f35cb624e08a4c935ca21ea4708]) successfully deleted
W20260811 22:12:23.003438 13892 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:23.003748 13892 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:23.003795 13892 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:23.008184 13892 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:23.008275 13892 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:23.008339 13892 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:12:23.014506 13892 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:42583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=40177
--tserver_master_addrs=127.28.47.190:36885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13892
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:23.015883 13892 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:23.017078 13892 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:23.028012 13904 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:23.028033 13907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:23.028118 13892 server_base.cc:1034] running on GCE node
W20260811 22:12:23.028033 13903 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:23.028761 13892 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:23.029584 13892 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:23.030756 13892 hybrid_clock.cc:648] HybridClock initialized: now 1786486343030723 us; error 47 us; skew 500 ppm
Aug 11 22:12:23 dist-test-slave-vxj2 krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486343, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:23.033717 13892 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:12:23.034659 13892 webserver.cc:460] Webserver started at http://127.28.47.130:40177/ using document root <none> and password file <none>
I20260811 22:12:23.035280 13892 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:23.035396 13892 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:23.039655 13892 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:12:23.042191 13918 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:23.043892 13892 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:12:23.044071 13892 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "f86c0d3f5f4c43eb80366236476c9c2b"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "c740603e4c4f9db221bbcad308903ca5"
server_key_iv: "dff4eb2d421bcc710b85db24fbc4a06b"
server_key_version: "encryptionkey@0"
I20260811 22:12:23.044734 13892 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:23.063243 13892 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:23.067236 13892 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:23.067507 13892 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:23.068217 13892 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:23.069206 13892 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:23.069285 13892 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:23.069384 13892 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:23.069435 13892 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:23.080367 13892 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:42583
I20260811 22:12:23.080474 14054 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:42583 every 8 connection(s)
I20260811 22:12:23.081760 13892 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:23.084796 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 13892
I20260811 22:12:23.085386 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:37001
--local_ip_for_outbound_sockets=127.28.47.131
--tserver_master_addrs=127.28.47.190:36885
--webserver_port=45843
--webserver_interface=127.28.47.131
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 11 22:12:23 dist-test-slave-vxj2 krb5kdc[12974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486343, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:23.099676 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.083974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:58913 (local address 127.28.47.190:36885)
0811 22:12:23.084161 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:23.084166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:23.085086 (+ 920us) server_negotiation.cc:407] Connection header received
0811 22:12:23.086655 (+ 1569us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:23.086661 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:23.086738 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:23.086854 (+ 116us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:23.088648 (+ 1794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.089274 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.091240 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.091533 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.091636 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.095188 (+ 3552us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:23.095192 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:23.095194 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:23.095223 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:23.096918 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:23.097736 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:23.097739 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:23.097741 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:23.097845 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:23.098345 (+ 500us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:23.098348 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:23.098349 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:23.098556 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:23.098638 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:23.099383 (+ 745us) server_negotiation.cc:299] Negotiation successful
0811 22:12:23.099541 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:12:23.100795 14057 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.084374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:58913)
0811 22:12:23.084893 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:23.084948 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:23.086395 (+ 1447us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:23.087115 (+ 720us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:23.087128 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:23.087667 (+ 539us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:23.088431 (+ 764us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.088458 (+ 27us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.089417 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.089423 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.091069 (+ 1646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.091088 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.091750 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.091758 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.092033 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.092783 (+ 750us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:23.092813 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:23.094926 (+ 2113us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:23.097067 (+ 2141us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:23.097078 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:23.097101 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:23.097599 (+ 498us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:23.097963 (+ 364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:23.097969 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:23.097972 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:23.098212 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:23.098680 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:23.098690 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:23.099080 (+ 390us) client_negotiation.cc:769] Sending connection context
0811 22:12:23.099303 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 22:12:23.099679 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":135,"threads_started":1}
I20260811 22:12:23.102242 14055 heartbeater.cc:347] Connected to a master server at 127.28.47.190:36885
I20260811 22:12:23.102543 14055 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:23.103164 14055 heartbeater.cc:511] Master 127.28.47.190:36885 requested a full tablet report, sending...
I20260811 22:12:23.104502 13661 ts_manager.cc:194] Registered new tserver with Master: f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583)
I20260811 22:12:23.105378 13661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:58913
W20260811 22:12:23.216904 14059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:23.217276 14059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:23.217330 14059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:23.221932 14059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:23.222012 14059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:23.222077 14059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:12:23.228072 14059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:37001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=45843
--tserver_master_addrs=127.28.47.190:36885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:23.229458 14059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:23.230598 14059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:23.241614 14064 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:23.241688 14059 server_base.cc:1034] running on GCE node
W20260811 22:12:23.241600 14068 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:23.241600 14066 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:23.242383 14059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:23.243140 14059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:23.244387 14059 hybrid_clock.cc:648] HybridClock initialized: now 1786486343244348 us; error 58 us; skew 500 ppm
Aug 11 22:12:23 dist-test-slave-vxj2 krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486343, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:23.248044 14059 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:12:23.249271 14059 webserver.cc:460] Webserver started at http://127.28.47.131:45843/ using document root <none> and password file <none>
I20260811 22:12:23.249998 14059 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:23.250115 14059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:23.255304 14059 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:12:23.258075 14075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:23.260274 14059 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:12:23.260627 14059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "460ccdb7881b4417a9454af680232eca"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "71909dca7e64f0f9faa31c5c19e49f22"
server_key_iv: "a96bc17e97c5ff24d9dbad94d44d96fd"
server_key_version: "encryptionkey@0"
I20260811 22:12:23.261883 14059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:23.280902 14059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:23.284729 14059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:23.284960 14059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:23.285657 14059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:23.286600 14059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:23.286676 14059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:23.286751 14059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:23.286798 14059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:23.296674 14059 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:37001
I20260811 22:12:23.296700 14209 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:37001 every 8 connection(s)
I20260811 22:12:23.298143 14059 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:23.304664 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 14059
Aug 11 22:12:23 dist-test-slave-vxj2 krb5kdc[12974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486343, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:23.314916 14213 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.305591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57514 (local address 127.28.47.190:36885)
0811 22:12:23.305889 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:23.305894 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:23.306056 (+ 162us) server_negotiation.cc:407] Connection header received
0811 22:12:23.306195 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:23.306198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:23.306238 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:23.306321 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:23.307208 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.307738 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.308663 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.309002 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.309119 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.310922 (+ 1803us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:23.310925 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:23.310928 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:23.310957 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:23.312708 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:23.313294 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:23.313298 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:23.313300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:23.313406 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:23.313699 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:23.313704 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:23.313706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:23.313992 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:23.314151 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:23.314501 (+ 350us) server_negotiation.cc:299] Negotiation successful
0811 22:12:23.314665 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":129,"threads_started":1}
I20260811 22:12:23.316089 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.300423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:54775 (local address 127.28.47.190:36885)
0811 22:12:23.300598 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:23.300605 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:23.301691 (+ 1086us) server_negotiation.cc:407] Connection header received
0811 22:12:23.303378 (+ 1687us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:23.303382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:23.303439 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:23.303581 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:23.305604 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.306490 (+ 886us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.307785 (+ 1295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.308028 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.308141 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.311262 (+ 3121us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:23.311266 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:23.311277 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:23.311321 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:23.313210 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:23.313983 (+ 773us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:23.313987 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:23.313988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:23.314124 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:23.314655 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:23.314659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:23.314661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:23.314925 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:23.315014 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:23.315785 (+ 771us) server_negotiation.cc:299] Negotiation successful
0811 22:12:23.315938 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:12:23.317306 14212 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.300919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:54775)
0811 22:12:23.301503 (+ 584us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:23.301553 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:23.303097 (+ 1544us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:23.303812 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:23.303822 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:23.304451 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:23.305348 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.305379 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.306643 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.306646 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.307626 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.307637 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.308214 (+ 577us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.308219 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.308412 (+ 193us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.309119 (+ 707us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:23.309138 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:23.311058 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:23.313350 (+ 2292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:23.313361 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:23.313383 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:23.313827 (+ 444us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:23.314281 (+ 454us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:23.314283 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:23.314285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:23.314512 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:23.315050 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:23.315060 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:23.315445 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 22:12:23.315716 (+ 271us) client_negotiation.cc:240] Negotiation successful
0811 22:12:23.316156 (+ 440us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":197,"threads_started":1}
I20260811 22:12:23.318742 14210 heartbeater.cc:347] Connected to a master server at 127.28.47.190:36885
I20260811 22:12:23.319064 14210 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:23.319669 14210 heartbeater.cc:511] Master 127.28.47.190:36885 requested a full tablet report, sending...
I20260811 22:12:23.321012 13661 ts_manager.cc:194] Registered new tserver with Master: 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001)
I20260811 22:12:23.321691 13661 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:54775
I20260811 22:12:23.329689 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:12:23.334801 13890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.330475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57522 (local address 127.28.47.190:36885)
0811 22:12:23.330650 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:23.330655 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:23.330851 (+ 196us) server_negotiation.cc:407] Connection header received
0811 22:12:23.331072 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:23.331075 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:23.331189 (+ 114us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:23.331316 (+ 127us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:23.331580 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.332117 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.333195 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.333432 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.333545 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.333930 (+ 385us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:23.334394 (+ 464us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:23.334535 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:23.334577 (+ 42us) server_negotiation.cc:299] Negotiation successful
0811 22:12:23.334639 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260811 22:12:23.335676 13661 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:57522:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:12:23.339393 13661 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:12:23.349134 13811 tablet_service.cc:1467] Processing CreateTablet for tablet eaff931eba8945b5be654df0605a2b18 (DEFAULT_TABLE table=test-table [id=ce1e1e05102f4c5a80694c72be193720]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:23.350126 13811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaff931eba8945b5be654df0605a2b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:23.353894 13895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.347656 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:42583 (local address 127.0.0.1:50952)
0811 22:12:23.347820 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:23.347839 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:23.347938 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:23.349103 (+ 1165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:23.349107 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:23.349129 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:23.349305 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.349313 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.350917 (+ 1604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.350921 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.352039 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.352052 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.353423 (+ 1371us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.353427 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.353544 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.353568 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:12:23.353627 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 22:12:23.353721 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":1}
I20260811 22:12:23.353912 13761 tablet_bootstrap.cc:492] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0: Bootstrap starting.
I20260811 22:12:23.354198 14221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.347955 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:37001 (local address 127.0.0.1:46772)
0811 22:12:23.348351 (+ 396us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:23.348368 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:23.348467 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:23.349278 (+ 811us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:23.349282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:23.349297 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:23.349427 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.349434 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.351026 (+ 1592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.351032 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.351824 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.351836 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.353437 (+ 1601us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.353440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.353868 (+ 428us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.353890 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:12:23.353968 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:12:23.354049 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"mutex_wait_us":317,"thread_start_us":146,"threads_started":1}
I20260811 22:12:23.354640 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.348261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46772 (local address 127.28.47.131:37001)
0811 22:12:23.348741 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:23.348751 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:23.348772 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:12:23.348840 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:23.348844 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:23.348994 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:23.349604 (+ 610us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:23.349781 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.350866 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.351995 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.353240 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.353523 (+ 283us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.353744 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:23.354382 (+ 638us) server_negotiation.cc:299] Negotiation successful
0811 22:12:23.354499 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":378,"thread_start_us":142,"threads_started":1}
I20260811 22:12:23.354686 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.347988 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50952 (local address 127.28.47.130:42583)
0811 22:12:23.348502 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:23.348516 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:23.348543 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:12:23.348632 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:23.348639 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:23.348887 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:23.349121 (+ 234us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:23.349473 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.350744 (+ 1271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.352194 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.353256 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.354210 (+ 954us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.354399 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:23.354450 (+ 51us) server_negotiation.cc:299] Negotiation successful
0811 22:12:23.354543 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":405,"thread_start_us":146,"threads_started":1}
I20260811 22:12:23.354943 13761 tablet_bootstrap.cc:654] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:23.356276 13761 tablet_bootstrap.cc:492] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0: No bootstrap required, opened a new log
I20260811 22:12:23.356387 13761 ts_tablet_manager.cc:1397] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260811 22:12:23.357450 13761 raft_consensus.cc:348] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:23.357726 13761 raft_consensus.cc:374] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:23.357839 13761 raft_consensus.cc:729] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc33f14b2a27475aa5f5dd7c49b63bd0, State: Initialized, Role: FOLLOWER
I20260811 22:12:23.358003 13761 consensus_queue.cc:260] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:23.358295 13761 ts_tablet_manager.cc:1428] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260811 22:12:23.358110 14130 tablet_service.cc:1467] Processing CreateTablet for tablet eaff931eba8945b5be654df0605a2b18 (DEFAULT_TABLE table=test-table [id=ce1e1e05102f4c5a80694c72be193720]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:23.358323 13978 tablet_service.cc:1467] Processing CreateTablet for tablet eaff931eba8945b5be654df0605a2b18 (DEFAULT_TABLE table=test-table [id=ce1e1e05102f4c5a80694c72be193720]), partition=RANGE (key) PARTITION UNBOUNDED
W20260811 22:12:23.359242 13889 tablet.cc:2369] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:12:23.359359 14130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaff931eba8945b5be654df0605a2b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:23.359472 13978 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet eaff931eba8945b5be654df0605a2b18. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:23.367534 14225 tablet_bootstrap.cc:492] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca: Bootstrap starting.
I20260811 22:12:23.367654 14226 tablet_bootstrap.cc:492] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: Bootstrap starting.
I20260811 22:12:23.370343 14226 tablet_bootstrap.cc:654] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:23.370343 14225 tablet_bootstrap.cc:654] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:23.371264 14226 log.cc:824] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:23.371263 14225 log.cc:824] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:23.374141 14226 tablet_bootstrap.cc:492] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: No bootstrap required, opened a new log
I20260811 22:12:23.374141 14225 tablet_bootstrap.cc:492] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca: No bootstrap required, opened a new log
I20260811 22:12:23.374394 14225 ts_tablet_manager.cc:1397] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:12:23.374394 14226 ts_tablet_manager.cc:1397] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:12:23.378882 14226 raft_consensus.cc:348] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:23.378882 14225 raft_consensus.cc:348] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:23.379240 14225 raft_consensus.cc:374] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:23.379240 14226 raft_consensus.cc:374] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:23.379302 14226 raft_consensus.cc:729] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f86c0d3f5f4c43eb80366236476c9c2b, State: Initialized, Role: FOLLOWER
I20260811 22:12:23.379302 14225 raft_consensus.cc:729] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 460ccdb7881b4417a9454af680232eca, State: Initialized, Role: FOLLOWER
I20260811 22:12:23.379848 14226 consensus_queue.cc:260] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:23.379848 14225 consensus_queue.cc:260] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:23.380779 14210 heartbeater.cc:503] Master 127.28.47.190:36885 was elected leader, sending a full tablet report...
I20260811 22:12:23.381284 14225 ts_tablet_manager.cc:1428] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:12:23.386039 14055 heartbeater.cc:503] Master 127.28.47.190:36885 was elected leader, sending a full tablet report...
I20260811 22:12:23.386257 14226 ts_tablet_manager.cc:1428] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
W20260811 22:12:23.549458 14211 tablet.cc:2369] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:12:23.583177 14056 tablet.cc:2369] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:12:23.675379 13894 raft_consensus.cc:482] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:23.675685 13894 raft_consensus.cc:504] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:23.676988 13894 leader_election.cc:288] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583), 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001)
I20260811 22:12:23.681670 14223 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.677503 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:53669 (local address 127.28.47.131:37001)
0811 22:12:23.677645 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:23.677650 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:23.677828 (+ 178us) server_negotiation.cc:407] Connection header received
0811 22:12:23.678020 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:23.678023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:23.678065 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:23.678154 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:23.678459 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.679099 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.680426 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.680850 (+ 424us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.681012 (+ 162us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.681074 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:23.681422 (+ 348us) server_negotiation.cc:299] Negotiation successful
0811 22:12:23.681528 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260811 22:12:23.681670 14222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.677470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:43437 (local address 127.28.47.130:42583)
0811 22:12:23.677643 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:23.677648 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:23.677885 (+ 237us) server_negotiation.cc:407] Connection header received
0811 22:12:23.678045 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:23.678049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:23.678102 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:23.678192 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:23.678524 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.679122 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.680458 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.680880 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.681038 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.681101 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:23.681422 (+ 321us) server_negotiation.cc:299] Negotiation successful
0811 22:12:23.681532 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:12:23.681696 14233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.677334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:42583 (local address 127.28.47.129:43437)
0811 22:12:23.677772 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:23.677784 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:23.677928 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:23.678220 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:23.678224 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:23.678241 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:23.678396 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.678402 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.679247 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.679251 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.680276 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.680289 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.681016 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.681020 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.681219 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.681271 (+ 52us) client_negotiation.cc:769] Sending connection context
0811 22:12:23.681355 (+ 84us) client_negotiation.cc:240] Negotiation successful
0811 22:12:23.681504 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":380,"mutex_wait_us":197,"thread_start_us":73,"threads_started":1}
I20260811 22:12:23.681710 14232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.677334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:37001 (local address 127.28.47.129:53669)
0811 22:12:23.677720 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:23.677732 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:23.677890 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:23.678170 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:23.678173 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:23.678189 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:23.678327 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.678333 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.679215 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.679219 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.680289 (+ 1070us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:23.680300 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.680998 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.681003 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:23.681182 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.681270 (+ 88us) client_negotiation.cc:769] Sending connection context
0811 22:12:23.681355 (+ 85us) client_negotiation.cc:240] Negotiation successful
0811 22:12:23.681504 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":120,"thread_start_us":101,"threads_started":1}
I20260811 22:12:23.682672 14001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaff931eba8945b5be654df0605a2b18" candidate_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" is_pre_election: true
I20260811 22:12:23.682668 14155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaff931eba8945b5be654df0605a2b18" candidate_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460ccdb7881b4417a9454af680232eca" is_pre_election: true
I20260811 22:12:23.683020 14155 raft_consensus.cc:2454] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc33f14b2a27475aa5f5dd7c49b63bd0 in term 0.
I20260811 22:12:23.683032 14001 raft_consensus.cc:2454] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate bc33f14b2a27475aa5f5dd7c49b63bd0 in term 0.
I20260811 22:12:23.683539 13757 leader_election.cc:302] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 460ccdb7881b4417a9454af680232eca, bc33f14b2a27475aa5f5dd7c49b63bd0; no voters:
I20260811 22:12:23.683933 13894 raft_consensus.cc:2781] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:23.684024 13894 raft_consensus.cc:482] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:23.684056 13894 raft_consensus.cc:3037] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:23.685297 13894 raft_consensus.cc:504] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:23.685746 13894 leader_election.cc:288] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 1 election: Requested vote from peers f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583), 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001)
I20260811 22:12:23.686342 14001 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaff931eba8945b5be654df0605a2b18" candidate_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f86c0d3f5f4c43eb80366236476c9c2b"
I20260811 22:12:23.686372 14155 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "eaff931eba8945b5be654df0605a2b18" candidate_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "460ccdb7881b4417a9454af680232eca"
I20260811 22:12:23.686530 14001 raft_consensus.cc:3037] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:23.686583 14155 raft_consensus.cc:3037] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:23.687821 14001 raft_consensus.cc:2454] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc33f14b2a27475aa5f5dd7c49b63bd0 in term 1.
I20260811 22:12:23.687847 14155 raft_consensus.cc:2454] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate bc33f14b2a27475aa5f5dd7c49b63bd0 in term 1.
I20260811 22:12:23.688279 13754 leader_election.cc:302] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: bc33f14b2a27475aa5f5dd7c49b63bd0, f86c0d3f5f4c43eb80366236476c9c2b; no voters:
I20260811 22:12:23.688513 13894 raft_consensus.cc:2781] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:23.688778 13894 raft_consensus.cc:686] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 LEADER]: Becoming Leader. State: Replica: bc33f14b2a27475aa5f5dd7c49b63bd0, State: Running, Role: LEADER
I20260811 22:12:23.689116 13894 consensus_queue.cc:237] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:23.692655 13661 catalog_manager.cc:5495] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 reported cstate change: term changed from 0 to 1, leader changed from <none> to bc33f14b2a27475aa5f5dd7c49b63bd0 (127.28.47.129). New cstate: current_term: 1 leader_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } health_report { overall_health: HEALTHY } } }
I20260811 22:12:23.725034 14238 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.720932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40404 (local address 127.28.47.129:42677)
0811 22:12:23.721312 (+ 380us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:23.721319 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:23.721332 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:12:23.721423 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:23.721425 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:23.721482 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:23.721626 (+ 144us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:23.721970 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.722550 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.723404 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:23.723645 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:23.723761 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:23.724087 (+ 326us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:23.724587 (+ 500us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:23.724723 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:23.724783 (+ 60us) server_negotiation.cc:299] Negotiation successful
0811 22:12:23.724847 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":118,"threads_started":1}
I20260811 22:12:23.729727 14155 raft_consensus.cc:1261] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca [term 1 FOLLOWER]: Refusing update from remote peer bc33f14b2a27475aa5f5dd7c49b63bd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:23.729725 14001 raft_consensus.cc:1261] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b [term 1 FOLLOWER]: Refusing update from remote peer bc33f14b2a27475aa5f5dd7c49b63bd0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:23.730769 14234 consensus_queue.cc:1035] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:23.731053 13894 consensus_queue.cc:1035] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:23.748785 13661 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:57522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:23.749017 13661 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:57522:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:23.751952 13661 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Sending DeleteTablet for 3 replicas of tablet eaff931eba8945b5be654df0605a2b18
I20260811 22:12:23.753047 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 13713
I20260811 22:12:23.753126 13978 tablet_service.cc:1514] Processing DeleteTablet for tablet eaff931eba8945b5be654df0605a2b18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:23 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:50952
I20260811 22:12:23.753800 14130 tablet_service.cc:1514] Processing DeleteTablet for tablet eaff931eba8945b5be654df0605a2b18 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:23 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:46772
I20260811 22:12:23.754314 14252 tablet_replica.cc:330] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca: stopping tablet replica
I20260811 22:12:23.754707 14252 raft_consensus.cc:2229] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:23.755077 14252 raft_consensus.cc:2258] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:23.761632 14251 tablet_replica.cc:330] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: stopping tablet replica
I20260811 22:12:23.762156 14251 raft_consensus.cc:2229] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:23.762629 14251 raft_consensus.cc:2258] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:23.763080 14252 ts_tablet_manager.cc:1905] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:23.764510 14251 ts_tablet_manager.cc:1905] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 22:12:23.765733 13649 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:42677 (error 108)
I20260811 22:12:23.766276 14252 ts_tablet_manager.cc:1918] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:23.766671 14252 log.cc:1196] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/eaff931eba8945b5be654df0605a2b18
I20260811 22:12:23.766876 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 13892
I20260811 22:12:23.767750 14221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.766699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:42677 (local address 127.0.0.1:40414)
0811 22:12:23.767535 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:23.767628 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:42677: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":717}
W20260811 22:12:23.768047 13649 catalog_manager.cc:4584] TS bc33f14b2a27475aa5f5dd7c49b63bd0 (127.28.47.129:42677): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet eaff931eba8945b5be654df0605a2b18: Network error: Client connection negotiation failed: client connection to 127.28.47.129:42677: connect: Connection refused (error 111)
I20260811 22:12:23.769294 14252 ts_tablet_manager.cc:1939] T eaff931eba8945b5be654df0605a2b18 P 460ccdb7881b4417a9454af680232eca: Deleting consensus metadata
I20260811 22:12:23.771152 13649 catalog_manager.cc:4841] TS 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001): tablet eaff931eba8945b5be654df0605a2b18 (table test-table [id=ce1e1e05102f4c5a80694c72be193720]) successfully deleted
I20260811 22:12:23.776423 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 14059
I20260811 22:12:23.781934 14221 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:23.776673 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:42583 (local address 127.0.0.1:50966)
0811 22:12:23.781669 (+ 4996us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:23.781787 (+ 118us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.130:42583: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4832}
I20260811 22:12:23.791414 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 13624
I20260811 22:12:23.803471 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:36885
--webserver_interface=127.28.47.190
--webserver_port=46509
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:36885
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:23.923485 14254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:23.923869 14254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:23.923972 14254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:23.931434 14254 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:23.931545 14254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:23.931573 14254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:23.931598 14254 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:23.931622 14254 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:23.939226 14254 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:36885
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:36885
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=46509
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:23.940402 14254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:23.941602 14254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:23.951411 14259 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:23.951416 14262 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:23.951455 14260 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:23.951634 14254 server_base.cc:1034] running on GCE node
I20260811 22:12:23.952515 14254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:23.953981 14254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:23.955219 14254 hybrid_clock.cc:648] HybridClock initialized: now 1786486343955195 us; error 40 us; skew 500 ppm
Aug 11 22:12:23 dist-test-slave-vxj2 krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486343, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:23.958614 14254 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:12:23.959710 14254 webserver.cc:460] Webserver started at http://127.28.47.190:46509/ using document root <none> and password file <none>
I20260811 22:12:23.960356 14254 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:23.960456 14254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:23.964917 14254 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:12:23.967625 14269 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:23.969105 14254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:12:23.969327 14254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "805a0aed16e246bc8c7228ea66c2d766"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "7235c6e74a54b7c367e4e87a2759d12d"
server_key_iv: "b31411d801e9ee364fd8f478cbaedca1"
server_key_version: "encryptionkey@0"
I20260811 22:12:23.970014 14254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:23.987215 14254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:23.990757 14254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:23.990993 14254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:23.998957 14254 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:36885
I20260811 22:12:23.999301 14342 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:36885 every 8 connection(s)
I20260811 22:12:24.000250 14254 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:24.001863 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 14254
I20260811 22:12:24.002377 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:42677
--local_ip_for_outbound_sockets=127.28.47.129
--tserver_master_addrs=127.28.47.190:36885
--webserver_port=33219
--webserver_interface=127.28.47.129
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:12:24.006188 14343 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:12:24.009272 14343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Bootstrap starting.
I20260811 22:12:24.035382 14343 log.cc:824] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:24.050585 14343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:12:24.051119 14343 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766: Bootstrap complete.
I20260811 22:12:24.057199 14343 raft_consensus.cc:348] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } }
I20260811 22:12:24.057879 14343 raft_consensus.cc:729] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 805a0aed16e246bc8c7228ea66c2d766, State: Initialized, Role: FOLLOWER
I20260811 22:12:24.058568 14343 consensus_queue.cc:260] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } }
I20260811 22:12:24.058717 14343 raft_consensus.cc:388] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:24.058825 14343 raft_consensus.cc:482] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:24.058956 14343 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:12:24.061314 14343 raft_consensus.cc:504] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } }
I20260811 22:12:24.061725 14343 leader_election.cc:302] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 805a0aed16e246bc8c7228ea66c2d766; no voters:
I20260811 22:12:24.062129 14343 leader_election.cc:288] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:12:24.062280 14347 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:12:24.062541 14347 raft_consensus.cc:686] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [term 3 LEADER]: Becoming Leader. State: Replica: 805a0aed16e246bc8c7228ea66c2d766, State: Running, Role: LEADER
I20260811 22:12:24.062894 14347 consensus_queue.cc:237] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } }
I20260811 22:12:24.063501 14343 sys_catalog.cc:564] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:24.064534 14349 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 805a0aed16e246bc8c7228ea66c2d766. Latest consensus state: current_term: 3 leader_uuid: "805a0aed16e246bc8c7228ea66c2d766" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } } }
I20260811 22:12:24.064669 14349 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:24.065106 14348 sys_catalog.cc:455] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "805a0aed16e246bc8c7228ea66c2d766" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "805a0aed16e246bc8c7228ea66c2d766" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36885 } } }
I20260811 22:12:24.065249 14348 sys_catalog.cc:458] T 00000000000000000000000000000000 P 805a0aed16e246bc8c7228ea66c2d766 [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:24.066105 14355 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:24.072391 14355 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:24.073374 14355 catalog_manager.cc:1259] Loaded cluster ID: 9efa9586523c4dcda91493e4e1d6bc5c
I20260811 22:12:24.073457 14355 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260811 22:12:24.074771 14355 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f5ba3fc59ad google::LogMessage::Fail() at ??:0
@ 0x7f5ba3fc888a google::LogMessage::SendToLog() at ??:0
@ 0x7f5ba3fc54de google::LogMessage::Flush() at ??:0
@ 0x7f5ba3fc6a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f5bae904fdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f5bae905d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f5bae902579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f5bae946640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5ba62173d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f5ba6217c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f5ba62193f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f5ba6208512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f5ba784c6db start_thread at ??:0
@ 0x7f5ba2ac361f clone at ??:0
W20260811 22:12:24.177292 14345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:24.177603 14345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:24.177655 14345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:24.182425 14345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:24.182514 14345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:24.182582 14345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:12:24.188784 14345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:42677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=33219
--tserver_master_addrs=127.28.47.190:36885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:24.190045 14345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:24.191170 14345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:24.201543 14391 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:24.201534 14392 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:24.201980 14345 server_base.cc:1034] running on GCE node
W20260811 22:12:24.201665 14395 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:24.202513 14345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:24.203265 14345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:24.204468 14345 hybrid_clock.cc:648] HybridClock initialized: now 1786486344204426 us; error 60 us; skew 500 ppm
Aug 11 22:12:24 dist-test-slave-vxj2 krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486344, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:24.207942 14345 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:12:24.209287 14345 webserver.cc:460] Webserver started at http://127.28.47.129:33219/ using document root <none> and password file <none>
I20260811 22:12:24.210098 14345 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:24.210204 14345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:24.215866 14345 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:12:24.218907 14406 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:24.220655 14345 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:12:24.220821 14345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "385295a90d342b06ecaaff51d7f80953"
server_key_iv: "211b1affd6a0ca0012fa16869ac8098a"
server_key_version: "encryptionkey@0"
I20260811 22:12:24.221655 14345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:24.267194 14345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:24.271562 14345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:24.271806 14345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:24.272635 14345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:24.274202 14413 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:12:24.278903 14345 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:12:24.279014 14345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 22:12:24.279096 14345 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:12:24.281628 14345 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:12:24.281709 14345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260811 22:12:24.282115 14413 tablet_bootstrap.cc:492] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0: Bootstrap starting.
I20260811 22:12:24.293424 14345 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:42677
I20260811 22:12:24.293813 14543 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:42677 every 8 connection(s)
I20260811 22:12:24.294770 14345 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:24.303069 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 14345
I20260811 22:12:24.303567 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:42583
--local_ip_for_outbound_sockets=127.28.47.130
--tserver_master_addrs=127.28.47.190:36885
--webserver_port=40177
--webserver_interface=127.28.47.130
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:12:24.303928 14413 log.cc:824] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:24.313488 14413 tablet_bootstrap.cc:492] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:12:24.313987 14413 tablet_bootstrap.cc:492] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0: Bootstrap complete.
I20260811 22:12:24.315706 14413 ts_tablet_manager.cc:1397] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0: Time spent bootstrapping tablet: real 0.034s user 0.016s sys 0.016s
I20260811 22:12:24.319226 14413 raft_consensus.cc:348] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:24.319873 14413 raft_consensus.cc:729] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: bc33f14b2a27475aa5f5dd7c49b63bd0, State: Initialized, Role: FOLLOWER
I20260811 22:12:24.320490 14413 consensus_queue.cc:260] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:24.321765 14413 ts_tablet_manager.cc:1428] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
W20260811 22:12:24.344063 14547 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:12:24.297546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:32881)
0811 22:12:24.298237 (+ 691us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:24.298303 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:24.299534 (+ 1231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:24.343246 (+ 43712us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":425,"thread_start_us":213,"threads_started":1}
W20260811 22:12:24.344630 14544 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:36885 (0 consecutive failures): Network error: Failed to ping master at 127.28.47.190:36885: Client connection negotiation failed: client connection to 127.28.47.190:36885: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:12:24.345594 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:24.345323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:52369)
0811 22:12:24.345459 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:24.345532 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:12:24.346364 14547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:24.346171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:40015)
0811 22:12:24.346272 (+ 101us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:24.346316 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260811 22:12:24.346613 14544 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:12:24.429402 14550 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:24.429693 14550 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:24.429741 14550 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:24.434070 14550 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:24.434149 14550 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:24.434211 14550 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:12:24.439826 14550 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:42583
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=40177
--tserver_master_addrs=127.28.47.190:36885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14550
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:24.441087 14550 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:24.442281 14550 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:24.453001 14566 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:24.453004 14563 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:24.453044 14564 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:24.453435 14550 server_base.cc:1034] running on GCE node
I20260811 22:12:24.453961 14550 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:24.454809 14550 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:24.456017 14550 hybrid_clock.cc:648] HybridClock initialized: now 1786486344455976 us; error 51 us; skew 500 ppm
Aug 11 22:12:24 dist-test-slave-vxj2 krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486344, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:24.459456 14550 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:12:24.460492 14550 webserver.cc:460] Webserver started at http://127.28.47.130:40177/ using document root <none> and password file <none>
I20260811 22:12:24.461290 14550 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:24.461393 14550 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:24.465782 14550 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:24.468374 14575 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:24.469859 14550 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:24.470027 14550 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "f86c0d3f5f4c43eb80366236476c9c2b"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "c740603e4c4f9db221bbcad308903ca5"
server_key_iv: "dff4eb2d421bcc710b85db24fbc4a06b"
server_key_version: "encryptionkey@0"
I20260811 22:12:24.470674 14550 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:24.491863 14550 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:24.495661 14550 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:24.495908 14550 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:24.496620 14550 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:24.497972 14582 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260811 22:12:24.502792 14582 ts_tablet_manager.cc:1852] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260811 22:12:24.503290 14582 ts_tablet_manager.cc:1905] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:24.508200 14582 ts_tablet_manager.cc:1918] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:24.508354 14582 log.cc:1196] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/eaff931eba8945b5be654df0605a2b18
I20260811 22:12:24.508788 14582 ts_tablet_manager.cc:1939] T eaff931eba8945b5be654df0605a2b18 P f86c0d3f5f4c43eb80366236476c9c2b: Deleting consensus metadata
I20260811 22:12:24.509281 14550 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260811 22:12:24.509373 14550 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.012s user 0.001s sys 0.000s
I20260811 22:12:24.509471 14550 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:24.509531 14550 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:24.519927 14550 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:42583
I20260811 22:12:24.519984 14714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:42583 every 8 connection(s)
I20260811 22:12:24.520957 14550 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:24.522408 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 14550
I20260811 22:12:24.522895 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:37001
--local_ip_for_outbound_sockets=127.28.47.131
--tserver_master_addrs=127.28.47.190:36885
--webserver_port=45843
--webserver_interface=127.28.47.131
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:33193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:12:24.525444 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:24.523733 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:60265)
0811 22:12:24.524291 (+ 558us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:24.524433 (+ 142us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":152,"threads_started":1}
W20260811 22:12:24.525974 14715 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:36885 (0 consecutive failures): Network error: Failed to ping master at 127.28.47.190:36885: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
I20260811 22:12:24.526712 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:24.526443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:59263)
0811 22:12:24.526584 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:24.526651 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260811 22:12:24.527357 14717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:24.527138 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:59351)
0811 22:12:24.527247 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:24.527297 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 22:12:24.527510 14715 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:12:24.646974 14718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:24.647333 14718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:24.647415 14718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:24.651969 14718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:24.652074 14718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:24.652175 14718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:12:24.657958 14718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:37001
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=45843
--tserver_master_addrs=127.28.47.190:36885
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:24.659220 14718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:24.660398 14718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:24.671388 14723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:24.671523 14718 server_base.cc:1034] running on GCE node
W20260811 22:12:24.671388 14724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:24.671388 14726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:24.672248 14718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:24.673059 14718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:24.674263 14718 hybrid_clock.cc:648] HybridClock initialized: now 1786486344674245 us; error 33 us; skew 500 ppm
Aug 11 22:12:24 dist-test-slave-vxj2 krb5kdc[12974](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486344, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:24.677459 14718 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:12:24.678539 14718 webserver.cc:460] Webserver started at http://127.28.47.131:45843/ using document root <none> and password file <none>
I20260811 22:12:24.679244 14718 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:24.679345 14718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:24.683900 14718 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260811 22:12:24.686371 14735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:24.687942 14718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:24.688138 14718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "460ccdb7881b4417a9454af680232eca"
format_stamp: "Formatted at 2026-08-11 22:12:21 on dist-test-slave-vxj2"
server_key: "71909dca7e64f0f9faa31c5c19e49f22"
server_key_iv: "a96bc17e97c5ff24d9dbad94d44d96fd"
server_key_version: "encryptionkey@0"
I20260811 22:12:24.688827 14718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:24.713096 14718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:24.716902 14718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:24.717144 14718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:24.717861 14718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:24.718811 14718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:24.718892 14718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:24.718966 14718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:24.719012 14718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:24.729416 14718 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:37001
I20260811 22:12:24.729436 14875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:37001 every 8 connection(s)
I20260811 22:12:24.730535 14718 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:24.731725 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 14718
I20260811 22:12:24.734782 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:24.732976 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:54501)
0811 22:12:24.733519 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:24.734016 (+ 497us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":182,"threads_started":1}
W20260811 22:12:24.735253 14877 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:36885 (0 consecutive failures): Network error: Failed to ping master at 127.28.47.190:36885: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
I20260811 22:12:24.735948 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:24.735666 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:50971)
0811 22:12:24.735823 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:24.735876 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260811 22:12:24.736696 14880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:24.736442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:39459)
0811 22:12:24.736585 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:24.736639 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260811 22:12:24.736963 14877 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:12:25.347872 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.347314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:35343)
0811 22:12:25.347647 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:25.347723 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":91,"threads_started":1}
I20260811 22:12:25.528609 14883 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.528096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:49759)
0811 22:12:25.528435 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:25.528496 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":75,"threads_started":1}
I20260811 22:12:25.551064 14884 raft_consensus.cc:482] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:25.551319 14884 raft_consensus.cc:504] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:25.552691 14884 leader_election.cc:288] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583), 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001)
Aug 11 22:12:25 dist-test-slave-vxj2 krb5kdc[12974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486344, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.131@KRBTEST.COM
Aug 11 22:12:25 dist-test-slave-vxj2 krb5kdc[12974](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486344, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.130@KRBTEST.COM
I20260811 22:12:25.569078 14881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.552978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:37001 (local address 127.28.47.129:59791)
0811 22:12:25.553180 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:25.553198 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:25.553318 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:25.555185 (+ 1867us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:25.555192 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:25.555785 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:25.556263 (+ 478us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.556280 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.557749 (+ 1469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.557751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.558812 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.558823 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.559804 (+ 981us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.559807 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.560004 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.560721 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:25.560739 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:25.562612 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:25.565375 (+ 2763us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.565381 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.565394 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.565931 (+ 537us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.566399 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.566403 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.566405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.566689 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.567613 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:25.567622 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:25.568088 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 22:12:25.568360 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 22:12:25.568919 (+ 559us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74,"mutex_wait_us":425,"spinlock_wait_cycles":26368}
I20260811 22:12:25.569128 14885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.553136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:59791 (local address 127.28.47.131:37001)
0811 22:12:25.553526 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:25.553559 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:25.553610 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:12:25.554383 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:25.554390 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:25.554770 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:25.555121 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:25.556468 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.557570 (+ 1102us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.558962 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.559635 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.559975 (+ 340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.562909 (+ 2934us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:25.562915 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:25.562928 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:25.562962 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:25.565240 (+ 2278us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.566085 (+ 845us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.566092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.566098 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.566269 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.566858 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.566866 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.566869 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.567430 (+ 561us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:25.567590 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:25.568577 (+ 987us) server_negotiation.cc:299] Negotiation successful
0811 22:12:25.568931 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":266,"thread_start_us":173,"threads_started":1}
I20260811 22:12:25.569180 14886 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.553265 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:32835 (local address 127.28.47.130:42583)
0811 22:12:25.553571 (+ 306us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:25.553581 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:25.553960 (+ 379us) server_negotiation.cc:407] Connection header received
0811 22:12:25.554747 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:25.554752 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:25.555034 (+ 282us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:25.555321 (+ 287us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:25.556466 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.557593 (+ 1127us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.558993 (+ 1400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.559604 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.559786 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.563242 (+ 3456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:25.563250 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:25.563268 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:25.563324 (+ 56us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:25.565387 (+ 2063us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.566085 (+ 698us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.566092 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.566098 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.566269 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:25.566806 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:25.566810 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:25.566812 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:25.567430 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:25.567590 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:25.568577 (+ 987us) server_negotiation.cc:299] Negotiation successful
0811 22:12:25.568968 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":74,"threads_started":1}
I20260811 22:12:25.569079 14887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.552982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:42583 (local address 127.28.47.129:32835)
0811 22:12:25.553822 (+ 840us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:25.553835 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:25.553981 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:25.555271 (+ 1290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:25.555274 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:25.555823 (+ 549us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:25.556262 (+ 439us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.556280 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.557742 (+ 1462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.557747 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.558836 (+ 1089us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:25.558851 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:25.559752 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:25.559759 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:25.559997 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:25.560765 (+ 768us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:25.560777 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:25.563018 (+ 2241us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:25.565498 (+ 2480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.565503 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.565508 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.565931 (+ 423us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.566399 (+ 468us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:25.566403 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:25.566405 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:25.566689 (+ 284us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:25.567613 (+ 924us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:25.567622 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:25.568088 (+ 466us) client_negotiation.cc:769] Sending connection context
0811 22:12:25.568360 (+ 272us) client_negotiation.cc:240] Negotiation successful
0811 22:12:25.568835 (+ 475us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":776,"mutex_wait_us":724,"spinlock_wait_cycles":37504,"thread_start_us":104,"threads_started":1}
W20260811 22:12:25.572196 14410 leader_election.cc:341] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001): Not found: Tablet not found: eaff931eba8945b5be654df0605a2b18
W20260811 22:12:25.572660 14407 leader_election.cc:341] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583): Not found: Tablet not found: eaff931eba8945b5be654df0605a2b18
I20260811 22:12:25.572773 14407 leader_election.cc:302] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc33f14b2a27475aa5f5dd7c49b63bd0; no voters: 460ccdb7881b4417a9454af680232eca, f86c0d3f5f4c43eb80366236476c9c2b
I20260811 22:12:25.573101 14884 raft_consensus.cc:2726] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:12:25.738312 14889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:25.737800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:53569)
0811 22:12:25.738102 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:25.738162 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":122,"threads_started":1}
I20260811 22:12:26.349551 14890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.348928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:43383)
0811 22:12:26.349365 (+ 437us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:26.349443 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":98,"threads_started":1}
I20260811 22:12:26.530143 14891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.529636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:37053)
0811 22:12:26.529955 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:26.530024 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":84,"threads_started":1}
I20260811 22:12:26.739853 14893 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:26.739303 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:33089)
0811 22:12:26.739633 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:26.739719 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":88,"threads_started":1}
I20260811 22:12:27.351045 14894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.350560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:55953)
0811 22:12:27.350859 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:27.350940 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":108,"threads_started":1}
I20260811 22:12:27.513514 14895 raft_consensus.cc:482] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:27.513664 14895 raft_consensus.cc:504] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:27.514176 14895 leader_election.cc:288] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583), 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001)
W20260811 22:12:27.515129 14410 leader_election.cc:341] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001): Not found: Tablet not found: eaff931eba8945b5be654df0605a2b18
W20260811 22:12:27.515311 14407 leader_election.cc:341] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583): Not found: Tablet not found: eaff931eba8945b5be654df0605a2b18
I20260811 22:12:27.515398 14407 leader_election.cc:302] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc33f14b2a27475aa5f5dd7c49b63bd0; no voters: 460ccdb7881b4417a9454af680232eca, f86c0d3f5f4c43eb80366236476c9c2b
I20260811 22:12:27.515596 14895 raft_consensus.cc:2726] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:12:27.531453 14897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.531008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:35401)
0811 22:12:27.531284 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:27.531345 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":79,"threads_started":1}
I20260811 22:12:27.741503 14898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:27.740906 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:49083)
0811 22:12:27.741267 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:27.741402 (+ 135us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":88,"threads_started":1}
I20260811 22:12:28.352587 14900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:28.352096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:55465)
0811 22:12:28.352417 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:28.352484 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":84,"threads_started":1}
I20260811 22:12:28.532835 14901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:28.532357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:50759)
0811 22:12:28.532664 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:28.532731 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":85,"threads_started":1}
I20260811 22:12:28.742883 14902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:28.742381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:53171)
0811 22:12:28.742711 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:28.742779 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":103,"threads_started":1}
I20260811 22:12:29.354236 14903 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.353739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:54425)
0811 22:12:29.354075 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:29.354137 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":84,"threads_started":1}
I20260811 22:12:29.534178 14904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.533744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:42041)
0811 22:12:29.534016 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:29.534076 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":85,"threads_started":1}
I20260811 22:12:29.744222 14905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:29.743786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:55569)
0811 22:12:29.744066 (+ 280us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:29.744124 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":203,"thread_start_us":81,"threads_started":1}
I20260811 22:12:30.259857 14906 raft_consensus.cc:482] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:30.259989 14906 raft_consensus.cc:504] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:30.260437 14906 leader_election.cc:288] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583), 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001)
W20260811 22:12:30.261389 14410 leader_election.cc:341] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001): Not found: Tablet not found: eaff931eba8945b5be654df0605a2b18
W20260811 22:12:30.261508 14407 leader_election.cc:341] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583): Not found: Tablet not found: eaff931eba8945b5be654df0605a2b18
I20260811 22:12:30.261564 14407 leader_election.cc:302] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc33f14b2a27475aa5f5dd7c49b63bd0; no voters: 460ccdb7881b4417a9454af680232eca, f86c0d3f5f4c43eb80366236476c9c2b
I20260811 22:12:30.261725 14906 raft_consensus.cc:2726] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:12:30.355684 14907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.355184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:52399)
0811 22:12:30.355504 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.355575 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":82,"threads_started":1}
I20260811 22:12:30.535616 14908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.535151 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:44537)
0811 22:12:30.535446 (+ 295us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.535515 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":79,"threads_started":1}
I20260811 22:12:30.745828 14910 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:30.745318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:38881)
0811 22:12:30.745658 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:30.745724 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":123,"threads_started":1}
I20260811 22:12:31.357259 14911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:31.356664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:45475)
0811 22:12:31.357005 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:31.357067 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":80,"threads_started":1}
I20260811 22:12:31.536939 14913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:31.536401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:44001)
0811 22:12:31.536773 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:31.536835 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":80,"threads_started":1}
I20260811 22:12:31.747498 14914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:31.746939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:45219)
0811 22:12:31.747293 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:31.747351 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":122,"threads_started":1}
I20260811 22:12:32.358770 14916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.358275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:43383)
0811 22:12:32.358601 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.358667 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":105,"threads_started":1}
I20260811 22:12:32.538836 14917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.538288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:32891)
0811 22:12:32.538644 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.538716 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":90,"threads_started":1}
I20260811 22:12:32.749004 14918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:32.748452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:42875)
0811 22:12:32.748823 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:32.748900 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":107,"threads_started":1}
I20260811 22:12:33.360159 14919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.359694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:49877)
0811 22:12:33.360001 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.360062 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":104,"threads_started":1}
I20260811 22:12:33.540343 14920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.539784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:42945)
0811 22:12:33.540142 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.540216 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":141,"threads_started":1}
I20260811 22:12:33.561271 14921 raft_consensus.cc:482] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:33.561430 14921 raft_consensus.cc:504] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:33.562026 14921 leader_election.cc:288] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583), 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001)
W20260811 22:12:33.563076 14407 leader_election.cc:341] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583): Not found: Tablet not found: eaff931eba8945b5be654df0605a2b18
W20260811 22:12:33.563203 14410 leader_election.cc:341] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001): Not found: Tablet not found: eaff931eba8945b5be654df0605a2b18
I20260811 22:12:33.563298 14410 leader_election.cc:302] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc33f14b2a27475aa5f5dd7c49b63bd0; no voters: 460ccdb7881b4417a9454af680232eca, f86c0d3f5f4c43eb80366236476c9c2b
I20260811 22:12:33.563470 14921 raft_consensus.cc:2726] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:12:33.750567 14922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:33.750018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:53661)
0811 22:12:33.750350 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:33.750425 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":93,"threads_started":1}
I20260811 22:12:34.361956 14923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:34.361370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:56921)
0811 22:12:34.361725 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:34.361798 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":92,"threads_started":1}
I20260811 22:12:34.542097 14924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:34.541506 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:37283)
0811 22:12:34.541924 (+ 418us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:34.541994 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":82,"threads_started":1}
I20260811 22:12:34.752038 14925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:34.751571 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:56429)
0811 22:12:34.751867 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:34.751935 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":85,"threads_started":1}
I20260811 22:12:35.363287 14926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:35.362804 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:54749)
0811 22:12:35.363124 (+ 320us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:35.363186 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":119,"threads_started":1}
I20260811 22:12:35.543561 14927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:35.543055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:53913)
0811 22:12:35.543378 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:35.543444 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":81,"threads_started":1}
I20260811 22:12:35.753700 14928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:35.753144 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:38377)
0811 22:12:35.753518 (+ 374us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:35.753587 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":125,"threads_started":1}
I20260811 22:12:35.854584 14930 raft_consensus.cc:482] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:35.854727 14930 raft_consensus.cc:504] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f86c0d3f5f4c43eb80366236476c9c2b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 42583 } } peers { permanent_uuid: "460ccdb7881b4417a9454af680232eca" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 37001 } } peers { permanent_uuid: "bc33f14b2a27475aa5f5dd7c49b63bd0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 42677 } }
I20260811 22:12:35.855254 14930 leader_election.cc:288] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583), 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001)
W20260811 22:12:35.856150 14410 leader_election.cc:341] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 460ccdb7881b4417a9454af680232eca (127.28.47.131:37001): Not found: Tablet not found: eaff931eba8945b5be654df0605a2b18
W20260811 22:12:35.856283 14407 leader_election.cc:341] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer f86c0d3f5f4c43eb80366236476c9c2b (127.28.47.130:42583): Not found: Tablet not found: eaff931eba8945b5be654df0605a2b18
I20260811 22:12:35.856381 14407 leader_election.cc:302] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: bc33f14b2a27475aa5f5dd7c49b63bd0; no voters: 460ccdb7881b4417a9454af680232eca, f86c0d3f5f4c43eb80366236476c9c2b
I20260811 22:12:35.856547 14930 raft_consensus.cc:2726] T eaff931eba8945b5be654df0605a2b18 P bc33f14b2a27475aa5f5dd7c49b63bd0 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:12:36.364862 14932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:36.364308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:43587)
0811 22:12:36.364645 (+ 337us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:36.364717 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":86,"threads_started":1}
I20260811 22:12:36.545257 14933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:36.544602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:59301)
0811 22:12:36.545015 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:36.545083 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":190,"threads_started":1}
I20260811 22:12:36.755169 14934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:36.754679 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:50001)
0811 22:12:36.754990 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:36.755064 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":85,"threads_started":1}
I20260811 22:12:37.366149 14935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.365669 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:33259)
0811 22:12:37.365984 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:37.366049 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":98,"threads_started":1}
I20260811 22:12:37.546753 14936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.546166 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:41269)
0811 22:12:37.546552 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:37.546637 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":296,"thread_start_us":160,"threads_started":1}
I20260811 22:12:37.756831 14937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:37.756213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:44867)
0811 22:12:37.756574 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:37.756647 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":105,"threads_started":1}
I20260811 22:12:38.367584 14938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.367099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:37007)
0811 22:12:38.367402 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:38.367470 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":85,"threads_started":1}
I20260811 22:12:38.548209 14939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.547695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:58435)
0811 22:12:38.548041 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:38.548106 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":140,"threads_started":1}
I20260811 22:12:38.758383 14940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:38.757894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:48143)
0811 22:12:38.758207 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:38.758279 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":103,"threads_started":1}
I20260811 22:12:39.369493 14941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.368860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.129:50637)
0811 22:12:39.369281 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:39.369373 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":131,"threads_started":1}
I20260811 22:12:39.549925 14942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.549432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.130:46307)
0811 22:12:39.549757 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:39.549819 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":238,"thread_start_us":99,"threads_started":1}
I20260811 22:12:39.740312 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 14345
I20260811 22:12:39.751137 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 14550
I20260811 22:12:39.760002 14943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:39.759512 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:36885 (local address 127.28.47.131:51345)
0811 22:12:39.759817 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:39.759897 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:36885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":108,"threads_started":1}
I20260811 22:12:39.764262 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 14718
2026-08-11T22:12:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (20163 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:12:39 dist-test-slave-vxj2 krb5kdc[14946](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:12:39 dist-test-slave-vxj2 krb5kdc[14946](info): set up 2 sockets
Aug 11 22:12:39 dist-test-slave-vxj2 krb5kdc[14946](info): commencing operation
krb5kdc: starting...
W20260811 22:12:40.952142 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.160s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:12:41 dist-test-slave-vxj2 krb5kdc[14946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:12:41Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:12:41Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:12:41.093788 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:38587
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:38587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:41.223809 14963 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:41.224167 14963 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:41.224212 14963 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:41.228622 14963 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:12:41.228717 14963 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:12:41.228734 14963 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:12:41.228749 14963 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:12:41.228770 14963 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:41.228789 14963 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:41.228806 14963 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:12:41.228998 14963 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:12:41.236047 14963 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:38587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:38587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14963
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:41.237440 14963 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:41.238634 14963 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:41.249015 14969 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:41.249039 14973 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:41.249280 14963 server_base.cc:1034] running on GCE node
W20260811 22:12:41.249023 14970 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:41.250157 14963 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:41.251675 14963 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:41.252923 14963 hybrid_clock.cc:648] HybridClock initialized: now 1786486361252894 us; error 49 us; skew 500 ppm
Aug 11 22:12:41 dist-test-slave-vxj2 krb5kdc[14946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486361, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:41.256136 14963 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:12:41.257318 14963 webserver.cc:460] Webserver started at http://127.28.47.190:39893/ using document root <none> and password file <none>
I20260811 22:12:41.257927 14963 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:41.257993 14963 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:41.258172 14963 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:41.260427 14963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "47bf86976000438eaa8a65d61307a11a"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "dcf62a0f7e96e0792b4b1bb5b6f42ffe"
server_key_iv: "0a9286c919562c1f77f4912af10d6db3"
server_key_version: "encryptionkey@0"
I20260811 22:12:41.261050 14963 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "47bf86976000438eaa8a65d61307a11a"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "dcf62a0f7e96e0792b4b1bb5b6f42ffe"
server_key_iv: "0a9286c919562c1f77f4912af10d6db3"
server_key_version: "encryptionkey@0"
I20260811 22:12:41.265645 14963 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260811 22:12:41.268582 14982 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:41.270160 14963 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:41.270321 14963 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "47bf86976000438eaa8a65d61307a11a"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "dcf62a0f7e96e0792b4b1bb5b6f42ffe"
server_key_iv: "0a9286c919562c1f77f4912af10d6db3"
server_key_version: "encryptionkey@0"
I20260811 22:12:41.270466 14963 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:41.287181 14963 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:41.291245 14963 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:41.291507 14963 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:41.299872 14963 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:38587
I20260811 22:12:41.300308 15041 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:38587 every 8 connection(s)
I20260811 22:12:41.301025 14963 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:12:41.302886 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 14963
I20260811 22:12:41.303099 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:12:41.303421 28862 external_mini_cluster.cc:1428] Setting key f6dc002554bcca530161319f9cde05d4
I20260811 22:12:41.304430 15042 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:41.311064 15042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Bootstrap starting.
Aug 11 22:12:41 dist-test-slave-vxj2 krb5kdc[14946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486361, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:41.313709 15042 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:41.314682 15042 log.cc:824] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:41.316908 15042 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: No bootstrap required, opened a new log
I20260811 22:12:41.319006 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:41.304993 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33218 (local address 127.28.47.190:38587)
0811 22:12:41.305558 (+ 565us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:41.305575 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:41.305617 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:12:41.306361 (+ 744us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:41.306380 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:41.306750 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:41.307195 (+ 445us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:41.308068 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:41.309330 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:41.310216 (+ 886us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:41.310707 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:41.310896 (+ 189us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:41.313681 (+ 2785us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:41.313687 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:41.313703 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:41.313745 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:41.315772 (+ 2027us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:41.316285 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:41.316290 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:41.316296 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:41.316417 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:41.316710 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:41.316715 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:41.316717 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:41.317325 (+ 608us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:41.317548 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:41.317851 (+ 303us) server_negotiation.cc:299] Negotiation successful
0811 22:12:41.318186 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":206,"threads_started":1}
I20260811 22:12:41.321400 15042 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } }
I20260811 22:12:41.321697 15042 raft_consensus.cc:374] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:41.321810 15042 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47bf86976000438eaa8a65d61307a11a, State: Initialized, Role: FOLLOWER
I20260811 22:12:41.322302 15042 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } }
I20260811 22:12:41.322451 15042 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:12:41.322496 15042 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:12:41.322573 15042 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:41.323663 15042 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } }
I20260811 22:12:41.324062 15042 leader_election.cc:302] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47bf86976000438eaa8a65d61307a11a; no voters:
I20260811 22:12:41.324493 15042 leader_election.cc:288] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:12:41.324628 15047 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:41.324934 15047 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 1 LEADER]: Becoming Leader. State: Replica: 47bf86976000438eaa8a65d61307a11a, State: Running, Role: LEADER
I20260811 22:12:41.325374 15047 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } }
I20260811 22:12:41.325990 15042 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:12:41.327247 15048 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "47bf86976000438eaa8a65d61307a11a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } } }
I20260811 22:12:41.327193 15049 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47bf86976000438eaa8a65d61307a11a. Latest consensus state: current_term: 1 leader_uuid: "47bf86976000438eaa8a65d61307a11a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } } }
I20260811 22:12:41.327387 15049 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:41.327387 15048 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: This master's current role is: LEADER
I20260811 22:12:41.327852 15056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:12:41.331449 15056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:12:41.338680 15056 catalog_manager.cc:1347] Generated new cluster ID: 75c130037f694db28c9841f1a71bacba
I20260811 22:12:41.338824 15056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:12:41.358585 15056 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:12:41.359674 15056 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:12:41.364789 15056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Generated new TSK 0
I20260811 22:12:41.365636 15056 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:12:41.405462 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38587
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:41.557039 15070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:41.557426 15070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:41.557477 15070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:41.561911 15070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:41.562002 15070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:41.562068 15070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:12:41.568619 15070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:41.569953 15070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:41.571087 15070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:41.582239 15081 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:41.582250 15077 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:41.582255 15079 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:41.582821 15070 server_base.cc:1034] running on GCE node
I20260811 22:12:41.583338 15070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:41.584123 15070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:41.585328 15070 hybrid_clock.cc:648] HybridClock initialized: now 1786486361585280 us; error 56 us; skew 500 ppm
Aug 11 22:12:41 dist-test-slave-vxj2 krb5kdc[14946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486361, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:41.588421 15070 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:12:41.589540 15070 webserver.cc:460] Webserver started at http://127.28.47.129:42857/ using document root <none> and password file <none>
I20260811 22:12:41.590122 15070 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:41.590186 15070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:41.590378 15070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:41.592687 15070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "fee780109c4d4afd81a3e7319de95ef4"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "1bb06e61ecebde41310686ebbb0e61e5"
server_key_iv: "6176e3b70680eea1f8d1ec2091d2da30"
server_key_version: "encryptionkey@0"
I20260811 22:12:41.593317 15070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "fee780109c4d4afd81a3e7319de95ef4"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "1bb06e61ecebde41310686ebbb0e61e5"
server_key_iv: "6176e3b70680eea1f8d1ec2091d2da30"
server_key_version: "encryptionkey@0"
I20260811 22:12:41.597558 15070 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:12:41.600324 15090 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:41.601945 15070 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:41.602131 15070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "fee780109c4d4afd81a3e7319de95ef4"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "1bb06e61ecebde41310686ebbb0e61e5"
server_key_iv: "6176e3b70680eea1f8d1ec2091d2da30"
server_key_version: "encryptionkey@0"
I20260811 22:12:41.602277 15070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:41.620950 15070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:41.625445 15070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:41.625694 15070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:41.626425 15070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:41.627462 15070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:41.627532 15070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:41.627602 15070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:41.627632 15070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:41.637784 15070 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:40159
I20260811 22:12:41.637823 15237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:40159 every 8 connection(s)
I20260811 22:12:41.638881 15070 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:12:41.645516 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 15070
I20260811 22:12:41.645674 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:12:41.645993 28862 external_mini_cluster.cc:1428] Setting key 319a444bc6c1f46b1b2cacc191244bcf
Aug 11 22:12:41 dist-test-slave-vxj2 krb5kdc[14946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486361, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:41.657094 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:41.641101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:45125 (local address 127.28.47.190:38587)
0811 22:12:41.641328 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:41.641333 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:41.642328 (+ 995us) server_negotiation.cc:407] Connection header received
0811 22:12:41.643685 (+ 1357us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:41.643689 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:41.643751 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:41.643843 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:41.645383 (+ 1540us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:41.646273 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:41.648113 (+ 1840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:41.648548 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:41.648683 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:41.651864 (+ 3181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:41.651870 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:41.651877 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:41.651918 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:41.653994 (+ 2076us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:41.654890 (+ 896us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:41.654893 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:41.654895 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:41.655020 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:41.655562 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:41.655565 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:41.655567 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:41.655833 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:41.655921 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:41.656764 (+ 843us) server_negotiation.cc:299] Negotiation successful
0811 22:12:41.656954 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":87}
I20260811 22:12:41.658476 15240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:41.641593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:45125)
0811 22:12:41.642152 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:41.642204 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:41.643425 (+ 1221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:41.644059 (+ 634us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:41.644066 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:41.644483 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:41.645191 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:41.645221 (+ 30us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:41.646469 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:41.646473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:41.647861 (+ 1388us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:41.647873 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:41.648792 (+ 919us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:41.648796 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:41.648967 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:41.649705 (+ 738us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:41.649724 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:41.651631 (+ 1907us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:41.654192 (+ 2561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:41.654203 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:41.654228 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:41.654754 (+ 526us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:41.655129 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:41.655135 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:41.655140 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:41.655414 (+ 274us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:41.655958 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:41.655971 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:41.656412 (+ 441us) client_negotiation.cc:769] Sending connection context
0811 22:12:41.656666 (+ 254us) client_negotiation.cc:240] Negotiation successful
0811 22:12:41.657109 (+ 443us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":147,"threads_started":1}
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
I20260811 22:12:41.659932 15238 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38587
I20260811 22:12:41.660240 15238 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:41.660859 15238 heartbeater.cc:511] Master 127.28.47.190:38587 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:12:41.662971 15002 ts_manager.cc:194] Registered new tserver with Master: fee780109c4d4afd81a3e7319de95ef4 (127.28.47.129:40159)
I20260811 22:12:41.664628 15002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:45125
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:12:41.681674 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38587
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:41.808830 15243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:41.809204 15243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:41.809266 15243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:41.813539 15243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:41.813619 15243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:41.813684 15243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:12:41.819804 15243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:41.821043 15243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:41.822259 15243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:41.833420 15250 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:41.833437 15254 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:41.833426 15252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:41.833846 15243 server_base.cc:1034] running on GCE node
I20260811 22:12:41.834465 15243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:41.835238 15243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:41.836470 15243 hybrid_clock.cc:648] HybridClock initialized: now 1786486361836410 us; error 83 us; skew 500 ppm
Aug 11 22:12:41 dist-test-slave-vxj2 krb5kdc[14946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486361, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:41.849751 15243 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:12:41.850903 15243 webserver.cc:460] Webserver started at http://127.28.47.130:42601/ using document root <none> and password file <none>
I20260811 22:12:41.851563 15243 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:41.851661 15243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:41.851913 15243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:41.854288 15243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "83417c0675ce4cb6a37e318208804c87"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "7d3b798a753038e62293be1152e76be4"
server_key_iv: "86d598879461a163634fc270f4ed67bb"
server_key_version: "encryptionkey@0"
I20260811 22:12:41.854923 15243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "83417c0675ce4cb6a37e318208804c87"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "7d3b798a753038e62293be1152e76be4"
server_key_iv: "86d598879461a163634fc270f4ed67bb"
server_key_version: "encryptionkey@0"
I20260811 22:12:41.859242 15243 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260811 22:12:41.862128 15262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:41.863528 15243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:12:41.863708 15243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "83417c0675ce4cb6a37e318208804c87"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "7d3b798a753038e62293be1152e76be4"
server_key_iv: "86d598879461a163634fc270f4ed67bb"
server_key_version: "encryptionkey@0"
I20260811 22:12:41.863849 15243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:41.876515 15243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:41.880731 15243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:41.880985 15243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:41.881908 15243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:41.883081 15243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:41.883141 15243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:41.883185 15243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:41.883201 15243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:41.893963 15243 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:41655
I20260811 22:12:41.893981 15406 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:41655 every 8 connection(s)
I20260811 22:12:41.894984 15243 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:12:41.900662 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 15243
I20260811 22:12:41.900818 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:12:41.901129 28862 external_mini_cluster.cc:1428] Setting key 571153a05f1a12cc08b9943b78cd41ce
Aug 11 22:12:41 dist-test-slave-vxj2 krb5kdc[14946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486361, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:12:41.912922 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:41.897256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:55883 (local address 127.28.47.190:38587)
0811 22:12:41.897435 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:41.897440 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:41.898471 (+ 1031us) server_negotiation.cc:407] Connection header received
0811 22:12:41.899936 (+ 1465us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:41.899939 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:41.899994 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:41.900078 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:41.901993 (+ 1915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:41.902610 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:41.904542 (+ 1932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:41.904979 (+ 437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:41.905092 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:41.908578 (+ 3486us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:41.908582 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:41.908585 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:41.908622 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:41.910466 (+ 1844us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:41.911176 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:41.911180 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:41.911182 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:41.911285 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:41.911672 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:41.911675 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:41.911677 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:41.911897 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:41.912004 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:41.912599 (+ 595us) server_negotiation.cc:299] Negotiation successful
0811 22:12:41.912745 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:12:41.913959 15409 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:41.897698 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:55883)
0811 22:12:41.898286 (+ 588us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:41.898339 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:41.899666 (+ 1327us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:41.900269 (+ 603us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:41.900277 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:41.900929 (+ 652us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:41.901754 (+ 825us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:41.901775 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:41.902758 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:41.902760 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:41.904357 (+ 1597us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:41.904375 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:41.905396 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:41.905399 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:41.905570 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:41.906256 (+ 686us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:41.906275 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:41.908352 (+ 2077us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:41.910655 (+ 2303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:41.910662 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:41.910684 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:41.911046 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:41.911383 (+ 337us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:41.911386 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:41.911388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:41.911561 (+ 173us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:41.912020 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:41.912030 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:41.912334 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 22:12:41.912537 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 22:12:41.912930 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"thread_start_us":142,"threads_started":1}
I20260811 22:12:41.915369 15407 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38587
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
I20260811 22:12:41.915696 15407 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:41.916347 15407 heartbeater.cc:511] Master 127.28.47.190:38587 requested a full tablet report, sending...
I20260811 22:12:41.917804 15002 ts_manager.cc:194] Registered new tserver with Master: 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655)
I20260811 22:12:41.918730 15002 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:55883
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:12:41.937345 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38587
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:12:42.062436 15413 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:12:42.062772 15413 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:12:42.062820 15413 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:12:42.067322 15413 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:12:42.067420 15413 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:12:42.067485 15413 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:12:42.074914 15413 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:38587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15413
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:12:42.076238 15413 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:12:42.077685 15413 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:12:42.089989 15425 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:42.089974 15423 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:12:42.089993 15421 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:12:42.090567 15413 server_base.cc:1034] running on GCE node
I20260811 22:12:42.091025 15413 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:12:42.091725 15413 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:12:42.092950 15413 hybrid_clock.cc:648] HybridClock initialized: now 1786486362092912 us; error 61 us; skew 500 ppm
Aug 11 22:12:42 dist-test-slave-vxj2 krb5kdc[14946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486362, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:12:42.096406 15413 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:12:42.097641 15413 webserver.cc:460] Webserver started at http://127.28.47.131:36687/ using document root <none> and password file <none>
I20260811 22:12:42.098302 15413 fs_manager.cc:359] Metadata directory not provided
I20260811 22:12:42.098371 15413 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:12:42.098559 15413 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:12:42.100986 15413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "8c1255f65a354397a1183c45130a455c"
format_stamp: "Formatted at 2026-08-11 22:12:42 on dist-test-slave-vxj2"
server_key: "4d321f1fa62ca027b21aaeacbc47e81e"
server_key_iv: "f294d142abf3a65c66482b03dad9180a"
server_key_version: "encryptionkey@0"
I20260811 22:12:42.101642 15413 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "8c1255f65a354397a1183c45130a455c"
format_stamp: "Formatted at 2026-08-11 22:12:42 on dist-test-slave-vxj2"
server_key: "4d321f1fa62ca027b21aaeacbc47e81e"
server_key_iv: "f294d142abf3a65c66482b03dad9180a"
server_key_version: "encryptionkey@0"
I20260811 22:12:42.106144 15413 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:12:42.109256 15433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:42.111001 15413 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:12:42.111186 15413 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "8c1255f65a354397a1183c45130a455c"
format_stamp: "Formatted at 2026-08-11 22:12:42 on dist-test-slave-vxj2"
server_key: "4d321f1fa62ca027b21aaeacbc47e81e"
server_key_iv: "f294d142abf3a65c66482b03dad9180a"
server_key_version: "encryptionkey@0"
I20260811 22:12:42.111349 15413 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:12:42.131704 15413 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:12:42.135650 15413 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:12:42.135941 15413 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:12:42.136654 15413 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:12:42.137684 15413 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:12:42.137769 15413 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:42.137866 15413 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:12:42.137910 15413 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:12:42.147929 15413 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:46445
I20260811 22:12:42.147980 15571 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:46445 every 8 connection(s)
I20260811 22:12:42.149013 15413 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:12:42.156415 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 15413
I20260811 22:12:42.156599 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:12:42.156891 28862 external_mini_cluster.cc:1428] Setting key 671835358c068a0d98308486966dc234
Aug 11 22:12:42 dist-test-slave-vxj2 krb5kdc[14946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486362, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:12:42.166257 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.151487 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:39833 (local address 127.28.47.190:38587)
0811 22:12:42.151681 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:42.151686 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:42.153104 (+ 1418us) server_negotiation.cc:407] Connection header received
0811 22:12:42.154175 (+ 1071us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:42.154178 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:42.154245 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:42.154352 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:42.155715 (+ 1363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.156387 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.158051 (+ 1664us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.158289 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.158412 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.162020 (+ 3608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:42.162024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:42.162026 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:42.162059 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:42.163710 (+ 1651us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:42.164398 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:42.164401 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:42.164402 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:42.164495 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:42.164896 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:42.164900 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:42.164907 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:42.165129 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:42.165294 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:42.165918 (+ 624us) server_negotiation.cc:299] Negotiation successful
0811 22:12:42.166073 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260811 22:12:42.167315 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.151882 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:39833)
0811 22:12:42.152909 (+ 1027us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:42.152965 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:42.153940 (+ 975us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:42.154556 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:42.154563 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:42.154985 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:12:42.155534 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:42.155550 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.156638 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.156643 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:42.157911 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:42.157923 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.158476 (+ 553us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.158480 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:42.158681 (+ 201us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.159686 (+ 1005us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:12:42.159705 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:12:42.161641 (+ 1936us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:12:42.163898 (+ 2257us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:42.163907 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:42.163928 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:42.164257 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:42.164603 (+ 346us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:12:42.164606 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:12:42.164608 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:12:42.164774 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:12:42.165306 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:12:42.165317 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:12:42.165624 (+ 307us) client_negotiation.cc:769] Sending connection context
0811 22:12:42.165851 (+ 227us) client_negotiation.cc:240] Negotiation successful
0811 22:12:42.166239 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":780,"thread_start_us":583,"threads_started":1}
I20260811 22:12:42.168915 15572 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38587
I20260811 22:12:42.169281 15572 heartbeater.cc:464] Registering TS with master...
I20260811 22:12:42.169936 15572 heartbeater.cc:511] Master 127.28.47.190:38587 requested a full tablet report, sending...
I20260811 22:12:42.171190 15001 ts_manager.cc:194] Registered new tserver with Master: 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445)
I20260811 22:12:42.171828 15001 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:39833
I20260811 22:12:42.183607 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:12:42.194521 15045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.185980 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33224 (local address 127.28.47.190:38587)
0811 22:12:42.186186 (+ 206us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:42.186191 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:42.186375 (+ 184us) server_negotiation.cc:407] Connection header received
0811 22:12:42.186565 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:42.186568 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:42.186622 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:42.186711 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:12:42.187516 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.188064 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.189009 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.189336 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.189453 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.191037 (+ 1584us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:12:42.191041 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:12:42.191043 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:12:42.191074 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:12:42.192616 (+ 1542us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:42.193112 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:42.193115 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:42.193117 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:42.193263 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:12:42.193543 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:12:42.193547 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:12:42.193549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:12:42.193826 (+ 277us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:12:42.193976 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:42.194210 (+ 234us) server_negotiation.cc:299] Negotiation successful
0811 22:12:42.194342 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:12:42.198024 15001 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33224:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:12:42.201327 15001 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:12:42.218344 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.211597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40656 (local address 127.28.47.130:41655)
0811 22:12:42.212687 (+ 1090us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:42.212701 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:42.212728 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:12:42.212816 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:42.212826 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:42.213069 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:42.213261 (+ 192us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:42.213618 (+ 357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.214827 (+ 1209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.216769 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.217471 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.217598 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.217810 (+ 212us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:42.218059 (+ 249us) server_negotiation.cc:299] Negotiation successful
0811 22:12:42.218167 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":969,"thread_start_us":146,"threads_started":1}
I20260811 22:12:42.218355 15588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.211824 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:40159 (local address 127.0.0.1:54780)
0811 22:12:42.212232 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:42.212265 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:42.212365 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:42.213644 (+ 1279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:42.213646 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:42.213668 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:42.213770 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:42.213776 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.214908 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.214915 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:42.216584 (+ 1669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:42.216596 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.217691 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.217694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:42.217824 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.217886 (+ 62us) client_negotiation.cc:769] Sending connection context
0811 22:12:42.217978 (+ 92us) client_negotiation.cc:240] Negotiation successful
0811 22:12:42.218148 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":86,"thread_start_us":109,"threads_started":1}
I20260811 22:12:42.218392 15589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.212198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:46445 (local address 127.0.0.1:47260)
0811 22:12:42.212779 (+ 581us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:42.212793 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:42.212883 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:42.213874 (+ 991us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:42.213877 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:42.213898 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:42.213992 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:42.213998 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.215408 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.215411 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:42.216513 (+ 1102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:42.216524 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.217878 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.217882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:42.217969 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.217993 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:12:42.218052 (+ 59us) client_negotiation.cc:240] Negotiation successful
0811 22:12:42.218265 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":510,"mutex_wait_us":143,"thread_start_us":96,"threads_started":1}
I20260811 22:12:42.218400 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.212727 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47260 (local address 127.28.47.131:46445)
0811 22:12:42.213130 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:42.213173 (+ 43us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:42.213193 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:12:42.213258 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:42.213262 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:42.213552 (+ 290us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:42.213692 (+ 140us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:42.214125 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.215258 (+ 1133us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.216676 (+ 1418us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.217542 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.217663 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.217815 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:42.218133 (+ 318us) server_negotiation.cc:299] Negotiation successful
0811 22:12:42.218248 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":144,"threads_started":1}
I20260811 22:12:42.219082 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.211413 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:41655 (local address 127.0.0.1:40656)
0811 22:12:42.212153 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:42.212212 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:42.212360 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:42.213249 (+ 889us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:42.213255 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:42.213278 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:42.213465 (+ 187us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:42.213475 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.215011 (+ 1536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.215014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:42.216665 (+ 1651us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:42.216675 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.217605 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.217609 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:42.217807 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.217892 (+ 85us) client_negotiation.cc:769] Sending connection context
0811 22:12:42.218020 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 22:12:42.218149 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":104,"thread_start_us":105,"threads_started":1}
I20260811 22:12:42.219744 15590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.212181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54780 (local address 127.28.47.129:40159)
0811 22:12:42.213092 (+ 911us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:42.213099 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:42.213126 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:12:42.213237 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:42.213243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:42.213481 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:42.213675 (+ 194us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:42.213912 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.214773 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.216708 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.217542 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.219275 (+ 1733us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.219447 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:42.219499 (+ 52us) server_negotiation.cc:299] Negotiation successful
0811 22:12:42.219600 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":143,"threads_started":1}
I20260811 22:12:42.222060 15329 tablet_service.cc:1467] Processing CreateTablet for tablet 54de7dd3659948498529679ca34b90e6 (DEFAULT_TABLE table=test-table [id=00c6577b87ec4b83836f277a42812deb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:42.222101 15496 tablet_service.cc:1467] Processing CreateTablet for tablet 54de7dd3659948498529679ca34b90e6 (DEFAULT_TABLE table=test-table [id=00c6577b87ec4b83836f277a42812deb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:42.222234 15147 tablet_service.cc:1467] Processing CreateTablet for tablet 54de7dd3659948498529679ca34b90e6 (DEFAULT_TABLE table=test-table [id=00c6577b87ec4b83836f277a42812deb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:12:42.223223 15329 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54de7dd3659948498529679ca34b90e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:42.223244 15496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54de7dd3659948498529679ca34b90e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:42.223238 15147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 54de7dd3659948498529679ca34b90e6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:12:42.230660 15592 tablet_bootstrap.cc:492] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Bootstrap starting.
I20260811 22:12:42.231760 15594 tablet_bootstrap.cc:492] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Bootstrap starting.
I20260811 22:12:42.232530 15593 tablet_bootstrap.cc:492] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c: Bootstrap starting.
I20260811 22:12:42.233942 15594 tablet_bootstrap.cc:654] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:42.234413 15592 tablet_bootstrap.cc:654] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:42.234771 15594 log.cc:824] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:42.235177 15593 tablet_bootstrap.cc:654] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c: Neither blocks nor log segments found. Creating new log.
I20260811 22:12:42.235361 15592 log.cc:824] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:42.236025 15593 log.cc:824] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:12:42.237133 15594 tablet_bootstrap.cc:492] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: No bootstrap required, opened a new log
I20260811 22:12:42.237414 15594 ts_tablet_manager.cc:1397] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.002s
I20260811 22:12:42.237524 15592 tablet_bootstrap.cc:492] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: No bootstrap required, opened a new log
I20260811 22:12:42.237726 15592 ts_tablet_manager.cc:1397] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:12:42.238126 15593 tablet_bootstrap.cc:492] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c: No bootstrap required, opened a new log
I20260811 22:12:42.238315 15593 ts_tablet_manager.cc:1397] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:12:42.241503 15594 raft_consensus.cc:348] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } }
I20260811 22:12:42.241817 15594 raft_consensus.cc:374] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:42.241899 15594 raft_consensus.cc:729] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fee780109c4d4afd81a3e7319de95ef4, State: Initialized, Role: FOLLOWER
I20260811 22:12:42.241912 15592 raft_consensus.cc:348] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } }
I20260811 22:12:42.242240 15592 raft_consensus.cc:374] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:42.242308 15592 raft_consensus.cc:729] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83417c0675ce4cb6a37e318208804c87, State: Initialized, Role: FOLLOWER
I20260811 22:12:42.242219 15593 raft_consensus.cc:348] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } }
I20260811 22:12:42.242518 15593 raft_consensus.cc:374] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:12:42.242458 15594 consensus_queue.cc:260] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } }
I20260811 22:12:42.242581 15593 raft_consensus.cc:729] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1255f65a354397a1183c45130a455c, State: Initialized, Role: FOLLOWER
I20260811 22:12:42.242830 15592 consensus_queue.cc:260] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } }
I20260811 22:12:42.242954 15593 consensus_queue.cc:260] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } }
I20260811 22:12:42.243400 15238 heartbeater.cc:503] Master 127.28.47.190:38587 was elected leader, sending a full tablet report...
I20260811 22:12:42.243930 15592 ts_tablet_manager.cc:1428] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260811 22:12:42.243930 15593 ts_tablet_manager.cc:1428] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260811 22:12:42.244349 15407 heartbeater.cc:503] Master 127.28.47.190:38587 was elected leader, sending a full tablet report...
I20260811 22:12:42.245100 15594 ts_tablet_manager.cc:1428] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Time spent starting tablet: real 0.008s user 0.004s sys 0.002s
I20260811 22:12:42.245330 15572 heartbeater.cc:503] Master 127.28.47.190:38587 was elected leader, sending a full tablet report...
I20260811 22:12:42.318646 15601 raft_consensus.cc:482] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:12:42.318965 15601 raft_consensus.cc:504] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } }
I20260811 22:12:42.320245 15601 leader_election.cc:288] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655), fee780109c4d4afd81a3e7319de95ef4 (127.28.47.129:40159)
I20260811 22:12:42.324397 15587 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.320684 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:35263 (local address 127.28.47.130:41655)
0811 22:12:42.320942 (+ 258us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:42.320947 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:42.321263 (+ 316us) server_negotiation.cc:407] Connection header received
0811 22:12:42.321418 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:42.321421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:42.321473 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:42.321559 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:42.321824 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.322323 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.323376 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.323796 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.323939 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.324000 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:42.324173 (+ 173us) server_negotiation.cc:299] Negotiation successful
0811 22:12:42.324243 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":143}
I20260811 22:12:42.324422 15590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.320712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:42551 (local address 127.28.47.129:40159)
0811 22:12:42.320900 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:42.320905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:42.320923 (+ 18us) server_negotiation.cc:407] Connection header received
0811 22:12:42.321068 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:42.321071 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:42.321117 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:42.321264 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:12:42.321567 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.322092 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.323366 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.323806 (+ 440us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.323954 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.324016 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:42.324211 (+ 195us) server_negotiation.cc:299] Negotiation successful
0811 22:12:42.324291 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:12:42.324419 15574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.320544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:40159 (local address 127.28.47.131:42551)
0811 22:12:42.320794 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:42.320812 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:42.320940 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:42.321290 (+ 350us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:42.321294 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:42.321315 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:42.321436 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:42.321441 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.322209 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.322214 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:42.323190 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:42.323201 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.323956 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.323960 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:42.324086 (+ 126us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.324103 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 22:12:42.324191 (+ 88us) client_negotiation.cc:240] Negotiation successful
0811 22:12:42.324267 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":161}
I20260811 22:12:42.324436 15604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.320598 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:41655 (local address 127.28.47.131:35263)
0811 22:12:42.321132 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:42.321143 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:12:42.321298 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:12:42.321569 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:12:42.321572 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:12:42.321588 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:12:42.321711 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:42.321716 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.322457 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.322460 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:42.323244 (+ 784us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:12:42.323254 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.323928 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.323932 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:12:42.324039 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.324068 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:12:42.324155 (+ 87us) client_negotiation.cc:240] Negotiation successful
0811 22:12:42.324236 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":261,"threads_started":1}
I20260811 22:12:42.325305 15353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54de7dd3659948498529679ca34b90e6" candidate_uuid: "8c1255f65a354397a1183c45130a455c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83417c0675ce4cb6a37e318208804c87" is_pre_election: true
I20260811 22:12:42.325299 15174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54de7dd3659948498529679ca34b90e6" candidate_uuid: "8c1255f65a354397a1183c45130a455c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fee780109c4d4afd81a3e7319de95ef4" is_pre_election: true
I20260811 22:12:42.325655 15353 raft_consensus.cc:2454] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c1255f65a354397a1183c45130a455c in term 0.
I20260811 22:12:42.325713 15174 raft_consensus.cc:2454] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8c1255f65a354397a1183c45130a455c in term 0.
I20260811 22:12:42.326252 15437 leader_election.cc:302] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 83417c0675ce4cb6a37e318208804c87, 8c1255f65a354397a1183c45130a455c, fee780109c4d4afd81a3e7319de95ef4; no voters:
I20260811 22:12:42.326658 15601 raft_consensus.cc:2781] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:12:42.326784 15601 raft_consensus.cc:482] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:12:42.326877 15601 raft_consensus.cc:3037] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:42.328193 15601 raft_consensus.cc:504] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } }
I20260811 22:12:42.328600 15601 leader_election.cc:288] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [CANDIDATE]: Term 1 election: Requested vote from peers 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655), fee780109c4d4afd81a3e7319de95ef4 (127.28.47.129:40159)
I20260811 22:12:42.329041 15353 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54de7dd3659948498529679ca34b90e6" candidate_uuid: "8c1255f65a354397a1183c45130a455c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "83417c0675ce4cb6a37e318208804c87"
I20260811 22:12:42.329226 15353 raft_consensus.cc:3037] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:42.329257 15174 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "54de7dd3659948498529679ca34b90e6" candidate_uuid: "8c1255f65a354397a1183c45130a455c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fee780109c4d4afd81a3e7319de95ef4"
I20260811 22:12:42.329424 15174 raft_consensus.cc:3037] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:12:42.330457 15353 raft_consensus.cc:2454] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1255f65a354397a1183c45130a455c in term 1.
I20260811 22:12:42.330531 15174 raft_consensus.cc:2454] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8c1255f65a354397a1183c45130a455c in term 1.
I20260811 22:12:42.330883 15437 leader_election.cc:302] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83417c0675ce4cb6a37e318208804c87, 8c1255f65a354397a1183c45130a455c; no voters:
I20260811 22:12:42.331099 15601 raft_consensus.cc:2781] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:12:42.331432 15601 raft_consensus.cc:686] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [term 1 LEADER]: Becoming Leader. State: Replica: 8c1255f65a354397a1183c45130a455c, State: Running, Role: LEADER
I20260811 22:12:42.331869 15601 consensus_queue.cc:237] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } }
I20260811 22:12:42.335772 14999 catalog_manager.cc:5495] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c reported cstate change: term changed from 0 to 1, leader changed from <none> to 8c1255f65a354397a1183c45130a455c (127.28.47.131). New cstate: current_term: 1 leader_uuid: "8c1255f65a354397a1183c45130a455c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } health_report { overall_health: UNKNOWN } } }
I20260811 22:12:42.347882 15591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.343820 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47268 (local address 127.28.47.131:46445)
0811 22:12:42.343947 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0811 22:12:42.343950 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:12:42.343964 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:12:42.344100 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:12:42.344102 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:12:42.344163 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:12:42.344274 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:12:42.344610 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.345297 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.346303 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:12:42.346555 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:12:42.346649 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:12:42.347005 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:12:42.347495 (+ 490us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:12:42.347590 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0811 22:12:42.347676 (+ 86us) server_negotiation.cc:299] Negotiation successful
0811 22:12:42.347739 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:12:42.354768 15353 raft_consensus.cc:1261] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [term 1 FOLLOWER]: Refusing update from remote peer 8c1255f65a354397a1183c45130a455c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:42.354768 15174 raft_consensus.cc:1261] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Refusing update from remote peer 8c1255f65a354397a1183c45130a455c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:12:42.355654 15605 consensus_queue.cc:1035] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [LEADER]: Connected to new peer: Peer: permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:42.355875 15601 consensus_queue.cc:1035] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [LEADER]: Connected to new peer: Peer: permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:12:42.364809 15610 mvcc.cc:205] Tried to move back new op lower bound from 7317448140196241408 to 7317448140114702336. Current Snapshot: MvccSnapshot[applied={T|T < 7317448140196241408}]
I20260811 22:12:42.365573 15612 mvcc.cc:205] Tried to move back new op lower bound from 7317448140196241408 to 7317448140114702336. Current Snapshot: MvccSnapshot[applied={T|T < 7317448140196241408}]
I20260811 22:12:42.374128 14998 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:42.374348 14998 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:33224:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:12:42.377060 14998 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Sending DeleteTablet for 3 replicas of tablet 54de7dd3659948498529679ca34b90e6
I20260811 22:12:42.378202 15329 tablet_service.cc:1514] Processing DeleteTablet for tablet 54de7dd3659948498529679ca34b90e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:42 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:40656
I20260811 22:12:42.378369 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 15070
I20260811 22:12:42.378309 15496 tablet_service.cc:1514] Processing DeleteTablet for tablet 54de7dd3659948498529679ca34b90e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:42 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:47260
I20260811 22:12:42.384526 15623 tablet_replica.cc:330] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: stopping tablet replica
I20260811 22:12:42.385025 15623 raft_consensus.cc:2229] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:12:42.385547 15623 raft_consensus.cc:2258] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:42.386040 15624 tablet_replica.cc:330] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c: stopping tablet replica
I20260811 22:12:42.386384 15624 raft_consensus.cc:2229] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:12:42.386816 15624 raft_consensus.cc:2258] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:12:42.388469 15624 ts_tablet_manager.cc:1905] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:42.391269 15624 ts_tablet_manager.cc:1918] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:12:42.391399 15624 log.cc:1196] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/54de7dd3659948498529679ca34b90e6
I20260811 22:12:42.391804 15624 ts_tablet_manager.cc:1939] T 54de7dd3659948498529679ca34b90e6 P 8c1255f65a354397a1183c45130a455c: Deleting consensus metadata
I20260811 22:12:42.393299 15623 ts_tablet_manager.cc:1905] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:12:42.393369 14986 catalog_manager.cc:4841] TS 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445): tablet 54de7dd3659948498529679ca34b90e6 (table test-table [id=00c6577b87ec4b83836f277a42812deb]) successfully deleted
W20260811 22:12:42.393853 14984 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:40159 (error 108)
I20260811 22:12:42.394140 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 15243
I20260811 22:12:42.405452 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.405003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:40159 (local address 127.0.0.1:54792)
0811 22:12:42.405264 (+ 261us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:42.405349 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260811 22:12:42.405802 14984 catalog_manager.cc:4584] TS fee780109c4d4afd81a3e7319de95ef4 (127.28.47.129:40159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 54de7dd3659948498529679ca34b90e6: Network error: Client connection negotiation failed: client connection to 127.28.47.129:40159: connect: Connection refused (error 111)
I20260811 22:12:42.412199 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 15413
I20260811 22:12:42.412513 15586 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:12:42.412132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:41655 (local address 127.0.0.1:40662)
0811 22:12:42.412325 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0811 22:12:42.412401 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.130:41655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":74}
I20260811 22:12:42.423033 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 14963
I20260811 22:13:02.435868 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:38587
--webserver_interface=127.28.47.190
--webserver_port=39893
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:38587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:02.556586 15627 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:02.556915 15627 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:02.556967 15627 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:02.561399 15627 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:13:02.561492 15627 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:13:02.561510 15627 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:13:02.561525 15627 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:02.561545 15627 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:02.561564 15627 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:02.561580 15627 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:02.561596 15627 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:02.567342 15627 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:38587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:38587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=39893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15627
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:02.568516 15627 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:02.569568 15627 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:02.584378 15633 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:02.584417 15637 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:02.584415 15634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:02.584498 15627 server_base.cc:1034] running on GCE node
I20260811 22:13:02.585592 15627 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:02.587054 15627 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:02.588286 15627 hybrid_clock.cc:648] HybridClock initialized: now 1786486382588252 us; error 45 us; skew 500 ppm
Aug 11 22:13:02 dist-test-slave-vxj2 krb5kdc[14946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486382, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:02.595623 15627 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:13:02.596747 15627 webserver.cc:460] Webserver started at http://127.28.47.190:39893/ using document root <none> and password file <none>
I20260811 22:13:02.597532 15627 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:02.597662 15627 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:02.602577 15627 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.003s
I20260811 22:13:02.605253 15646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:02.606788 15627 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:02.606984 15627 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "47bf86976000438eaa8a65d61307a11a"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "dcf62a0f7e96e0792b4b1bb5b6f42ffe"
server_key_iv: "0a9286c919562c1f77f4912af10d6db3"
server_key_version: "encryptionkey@0"
I20260811 22:13:02.607681 15627 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:02.627065 15627 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:02.630941 15627 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:02.631206 15627 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:02.639436 15627 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:38587
I20260811 22:13:02.639791 15705 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:38587 every 8 connection(s)
I20260811 22:13:02.640769 15627 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:02.644769 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 15627
I20260811 22:13:02.645370 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:40159
--local_ip_for_outbound_sockets=127.28.47.129
--tserver_master_addrs=127.28.47.190:38587
--webserver_port=42857
--webserver_interface=127.28.47.129
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:13:02.646176 15706 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:13:02.648180 15706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Bootstrap starting.
I20260811 22:13:02.666886 15706 log.cc:824] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:02.676488 15706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:13:02.676973 15706 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Bootstrap complete.
I20260811 22:13:02.681509 15706 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } }
I20260811 22:13:02.681988 15706 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47bf86976000438eaa8a65d61307a11a, State: Initialized, Role: FOLLOWER
I20260811 22:13:02.682514 15706 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } }
I20260811 22:13:02.682672 15706 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:02.682775 15706 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:02.682909 15706 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 1 FOLLOWER]: Advancing to term 2
I20260811 22:13:02.685401 15706 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } }
I20260811 22:13:02.685781 15706 leader_election.cc:302] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47bf86976000438eaa8a65d61307a11a; no voters:
I20260811 22:13:02.686156 15706 leader_election.cc:288] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260811 22:13:02.686280 15711 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 2 FOLLOWER]: Leader election won for term 2
I20260811 22:13:02.686517 15711 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 2 LEADER]: Becoming Leader. State: Replica: 47bf86976000438eaa8a65d61307a11a, State: Running, Role: LEADER
I20260811 22:13:02.686843 15711 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } }
I20260811 22:13:02.687620 15706 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:02.687938 15711 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47bf86976000438eaa8a65d61307a11a. Latest consensus state: current_term: 2 leader_uuid: "47bf86976000438eaa8a65d61307a11a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } } }
I20260811 22:13:02.688072 15711 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:02.688468 15717 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:02.688617 15714 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "47bf86976000438eaa8a65d61307a11a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } } }
I20260811 22:13:02.688827 15714 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:02.694137 15717 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:02.694974 15717 catalog_manager.cc:1259] Loaded cluster ID: 75c130037f694db28c9841f1a71bacba
I20260811 22:13:02.695067 15717 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:02.697032 15717 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:02.697852 15717 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Loaded TSK: 0
I20260811 22:13:02.713546 15717 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Generated new TSK 1
I20260811 22:13:02.713889 15717 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:13:02.780433 15709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:02.780776 15709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:02.780869 15709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:02.785310 15709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:02.785403 15709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:02.785533 15709 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:13:02.791592 15709 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:40159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=42857
--tserver_master_addrs=127.28.47.190:38587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:02.792739 15709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:02.793817 15709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:02.804325 15739 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:02.804281 15738 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:02.804280 15741 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:02.804575 15709 server_base.cc:1034] running on GCE node
I20260811 22:13:02.805023 15709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:02.805776 15709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:02.807013 15709 hybrid_clock.cc:648] HybridClock initialized: now 1786486382806980 us; error 45 us; skew 500 ppm
Aug 11 22:13:02 dist-test-slave-vxj2 krb5kdc[14946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486382, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:02.810065 15709 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:13:02.811125 15709 webserver.cc:460] Webserver started at http://127.28.47.129:42857/ using document root <none> and password file <none>
I20260811 22:13:02.811748 15709 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:02.811826 15709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:02.816233 15709 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:13:02.819060 15750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:02.820868 15709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:13:02.821049 15709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "fee780109c4d4afd81a3e7319de95ef4"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "1bb06e61ecebde41310686ebbb0e61e5"
server_key_iv: "6176e3b70680eea1f8d1ec2091d2da30"
server_key_version: "encryptionkey@0"
I20260811 22:13:02.821797 15709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:02.864212 15709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:02.868479 15709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:02.868755 15709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:02.869539 15709 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:02.870841 15761 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:13:02.875650 15709 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:13:02.875736 15709 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 22:13:02.875931 15709 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:13:02.877890 15709 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:13:02.877950 15709 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260811 22:13:02.878286 15761 tablet_bootstrap.cc:492] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Bootstrap starting.
I20260811 22:13:02.890177 15709 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:40159
I20260811 22:13:02.890211 15902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:40159 every 8 connection(s)
I20260811 22:13:02.891232 15709 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:13:02.894546 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 15709
I20260811 22:13:02.895105 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:41655
--local_ip_for_outbound_sockets=127.28.47.130
--tserver_master_addrs=127.28.47.190:38587
--webserver_port=42601
--webserver_interface=127.28.47.130
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:13:02.899405 15761 log.cc:824] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:13:02 dist-test-slave-vxj2 krb5kdc[14946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486382, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:13:02.909469 15761 tablet_bootstrap.cc:492] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:13:02.909965 15761 tablet_bootstrap.cc:492] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Bootstrap complete.
I20260811 22:13:02.911566 15761 ts_tablet_manager.cc:1397] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Time spent bootstrapping tablet: real 0.034s user 0.022s sys 0.009s
I20260811 22:13:02.913710 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:02.893719 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:35031 (local address 127.28.47.190:38587)
0811 22:13:02.894801 (+ 1082us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:02.894819 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:02.894870 (+ 51us) server_negotiation.cc:407] Connection header received
0811 22:13:02.895781 (+ 911us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:02.895792 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:02.896088 (+ 296us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:02.896420 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:02.897993 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.899587 (+ 1594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.901478 (+ 1891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.902202 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.902389 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:02.906497 (+ 4108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:02.906505 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:02.906525 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:02.906569 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:02.909185 (+ 2616us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:02.910114 (+ 929us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:02.910121 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:02.910127 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:02.910332 (+ 205us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:02.910778 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:02.910781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:02.910783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:02.911441 (+ 658us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:02.911596 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:02.912389 (+ 793us) server_negotiation.cc:299] Negotiation successful
0811 22:13:02.912722 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":933,"thread_start_us":150,"threads_started":1}
I20260811 22:13:02.913722 15906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:02.893672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:35031)
0811 22:13:02.894186 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:02.894241 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:02.895294 (+ 1053us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:02.896472 (+ 1178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:02.896480 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:02.896996 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:02.897762 (+ 766us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:02.897788 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.899773 (+ 1985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.899779 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:02.901308 (+ 1529us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:02.901327 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.902386 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.902393 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:02.902653 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:02.903454 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:02.903484 (+ 30us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:02.906138 (+ 2654us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:02.909363 (+ 3225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:02.909375 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:02.909392 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:02.909864 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:02.910457 (+ 593us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:02.910460 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:02.910463 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:02.910655 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:02.911572 (+ 917us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:02.911580 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:02.911947 (+ 367us) client_negotiation.cc:769] Sending connection context
0811 22:13:02.912304 (+ 357us) client_negotiation.cc:240] Negotiation successful
0811 22:13:02.912694 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":150,"threads_started":1}
I20260811 22:13:02.914877 15761 raft_consensus.cc:348] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } }
I20260811 22:13:02.915338 15761 raft_consensus.cc:729] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fee780109c4d4afd81a3e7319de95ef4, State: Initialized, Role: FOLLOWER
I20260811 22:13:02.915812 15761 consensus_queue.cc:260] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } }
I20260811 22:13:02.916819 15903 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38587
I20260811 22:13:02.916831 15761 ts_tablet_manager.cc:1428] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Time spent starting tablet: real 0.005s user 0.007s sys 0.000s
I20260811 22:13:02.917080 15903 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:02.917677 15903 heartbeater.cc:511] Master 127.28.47.190:38587 requested a full tablet report, sending...
I20260811 22:13:02.920392 15665 ts_manager.cc:194] Registered new tserver with Master: fee780109c4d4afd81a3e7319de95ef4 (127.28.47.129:40159)
I20260811 22:13:02.924443 15665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:35031
I20260811 22:13:02.927624 15903 heartbeater.cc:503] Master 127.28.47.190:38587 was elected leader, sending a full tablet report...
Aug 11 22:13:02 dist-test-slave-vxj2 krb5kdc[14946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486382, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for kudu/127.28.47.129@KRBTEST.COM
I20260811 22:13:02.936621 15912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:02.923884 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:40159 (local address 127.0.0.1:37812)
0811 22:13:02.924868 (+ 984us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:02.924931 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:02.925091 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:02.925746 (+ 655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:02.925752 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:02.926243 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:02.926389 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:02.926398 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.927856 (+ 1458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.927860 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:02.928817 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:02.928830 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.929723 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.929727 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:02.929864 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:02.930495 (+ 631us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:02.930507 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:02.932073 (+ 1566us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:02.934414 (+ 2341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:02.934428 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:02.934435 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:02.934751 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:02.935135 (+ 384us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:02.935138 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:02.935140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:02.935237 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:02.935822 (+ 585us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:02.935829 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:02.935981 (+ 152us) client_negotiation.cc:769] Sending connection context
0811 22:13:02.936145 (+ 164us) client_negotiation.cc:240] Negotiation successful
0811 22:13:02.936414 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":807,"thread_start_us":120,"threads_started":1}
I20260811 22:13:02.936621 15913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:02.924032 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37812 (local address 127.28.47.129:40159)
0811 22:13:02.924535 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:02.924570 (+ 35us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:02.925063 (+ 493us) server_negotiation.cc:407] Connection header received
0811 22:13:02.925333 (+ 270us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:02.925342 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:02.925566 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:02.925728 (+ 162us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:02.926531 (+ 803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.927626 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.928968 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:02.929590 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:02.929714 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:02.932276 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:02.932288 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:02.932301 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:02.932328 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:02.934242 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:02.934920 (+ 678us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:02.934926 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:02.934927 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:02.935022 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:02.935350 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:02.935354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:02.935355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:02.935664 (+ 309us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:02.935847 (+ 183us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:02.936253 (+ 406us) server_negotiation.cc:299] Negotiation successful
0811 22:13:02.936453 (+ 200us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":386,"thread_start_us":152,"threads_started":1}
I20260811 22:13:02.938200 15809 tablet_service.cc:1514] Processing DeleteTablet for tablet 54de7dd3659948498529679ca34b90e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:42 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:37812
I20260811 22:13:02.939339 15915 tablet_replica.cc:330] stopping tablet replica
I20260811 22:13:02.939736 15915 raft_consensus.cc:2229] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:02.940075 15915 raft_consensus.cc:2258] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:02.941260 15915 ts_tablet_manager.cc:1905] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:02.945374 15915 ts_tablet_manager.cc:1918] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:02.945518 15915 log.cc:1196] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal/wals/54de7dd3659948498529679ca34b90e6
I20260811 22:13:02.945863 15915 ts_tablet_manager.cc:1939] T 54de7dd3659948498529679ca34b90e6 P fee780109c4d4afd81a3e7319de95ef4: Deleting consensus metadata
I20260811 22:13:02.947868 15648 catalog_manager.cc:4841] TS fee780109c4d4afd81a3e7319de95ef4 (127.28.47.129:40159): tablet 54de7dd3659948498529679ca34b90e6 (table test-table [id=00c6577b87ec4b83836f277a42812deb]) successfully deleted
W20260811 22:13:03.029639 15908 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:03.029961 15908 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:03.030009 15908 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:03.034394 15908 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:03.034471 15908 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:03.034535 15908 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:13:03.040227 15908 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:41655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=42601
--tserver_master_addrs=127.28.47.190:38587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15908
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:03.041483 15908 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:03.042564 15908 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:03.053546 15920 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:03.053710 15908 server_base.cc:1034] running on GCE node
W20260811 22:13:03.053541 15923 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:03.053901 15921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:03.054375 15908 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:03.055202 15908 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:03.056384 15908 hybrid_clock.cc:648] HybridClock initialized: now 1786486383056362 us; error 36 us; skew 500 ppm
Aug 11 22:13:03 dist-test-slave-vxj2 krb5kdc[14946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486383, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:03.059547 15908 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:13:03.060628 15908 webserver.cc:460] Webserver started at http://127.28.47.130:42601/ using document root <none> and password file <none>
I20260811 22:13:03.061321 15908 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:03.061450 15908 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:03.065632 15908 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:13:03.068120 15932 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:03.069648 15908 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:03.069795 15908 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "83417c0675ce4cb6a37e318208804c87"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "7d3b798a753038e62293be1152e76be4"
server_key_iv: "86d598879461a163634fc270f4ed67bb"
server_key_version: "encryptionkey@0"
I20260811 22:13:03.070374 15908 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:03.087648 15908 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:03.091671 15908 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:03.091928 15908 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:03.092615 15908 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:03.093946 15939 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:13:03.098953 15908 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:13:03.099082 15908 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 22:13:03.099181 15908 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:13:03.101133 15908 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:13:03.101284 15908 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.002s
I20260811 22:13:03.101727 15939 tablet_bootstrap.cc:492] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Bootstrap starting.
I20260811 22:13:03.108685 15939 log.cc:824] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:03.112744 15908 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:41655
I20260811 22:13:03.112835 16078 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:41655 every 8 connection(s)
I20260811 22:13:03.113891 15908 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:13:03.115927 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 15908
I20260811 22:13:03.116425 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:46445
--local_ip_for_outbound_sockets=127.28.47.131
--tserver_master_addrs=127.28.47.190:38587
--webserver_port=36687
--webserver_interface=127.28.47.131
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:13:03.119690 15939 tablet_bootstrap.cc:492] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:13:03.120257 15939 tablet_bootstrap.cc:492] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Bootstrap complete.
I20260811 22:13:03.120692 15939 ts_tablet_manager.cc:1397] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Time spent bootstrapping tablet: real 0.019s user 0.017s sys 0.000s
Aug 11 22:13:03 dist-test-slave-vxj2 krb5kdc[14946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486383, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:13:03.125376 15939 raft_consensus.cc:348] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } }
I20260811 22:13:03.126235 15939 raft_consensus.cc:729] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83417c0675ce4cb6a37e318208804c87, State: Initialized, Role: FOLLOWER
I20260811 22:13:03.127018 15939 consensus_queue.cc:260] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } } peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } }
I20260811 22:13:03.128549 15939 ts_tablet_manager.cc:1428] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Time spent starting tablet: real 0.008s user 0.008s sys 0.000s
I20260811 22:13:03.130513 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.117202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:58289 (local address 127.28.47.190:38587)
0811 22:13:03.117355 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:03.117361 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:03.117381 (+ 20us) server_negotiation.cc:407] Connection header received
0811 22:13:03.118675 (+ 1294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:03.118679 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:03.118758 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:03.118869 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:03.120416 (+ 1547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.121039 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.122243 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.122491 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.122607 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.125949 (+ 3342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:03.125953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:03.125955 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:03.125984 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:03.128072 (+ 2088us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:03.128676 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:03.128682 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:03.128686 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:03.128802 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:03.129219 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:03.129222 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:03.129224 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:03.129461 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:03.129583 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:03.130158 (+ 575us) server_negotiation.cc:299] Negotiation successful
0811 22:13:03.130328 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:13:03.131407 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.116097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:58289)
0811 22:13:03.116949 (+ 852us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:03.117017 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:03.118414 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:03.119045 (+ 631us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:03.119053 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:03.119543 (+ 490us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:03.120187 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.120204 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.121230 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.121233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.122098 (+ 865us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.122108 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.122653 (+ 545us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.122658 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.122855 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.123940 (+ 1085us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:03.123958 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:03.125632 (+ 1674us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:03.128217 (+ 2585us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:03.128223 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:03.128240 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:03.128542 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:03.128909 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:03.128912 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:03.128914 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:03.129074 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:03.129585 (+ 511us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:03.129595 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:03.129885 (+ 290us) client_negotiation.cc:769] Sending connection context
0811 22:13:03.130096 (+ 211us) client_negotiation.cc:240] Negotiation successful
0811 22:13:03.130476 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":578,"thread_start_us":166,"threads_started":1}
I20260811 22:13:03.133230 16079 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38587
I20260811 22:13:03.133533 16079 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:03.134140 16079 heartbeater.cc:511] Master 127.28.47.190:38587 requested a full tablet report, sending...
I20260811 22:13:03.136168 15665 ts_manager.cc:194] Registered new tserver with Master: 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655)
I20260811 22:13:03.137777 15665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:58289
I20260811 22:13:03.140547 16079 heartbeater.cc:503] Master 127.28.47.190:38587 was elected leader, sending a full tablet report...
Aug 11 22:13:03 dist-test-slave-vxj2 krb5kdc[14946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486382, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for kudu/127.28.47.130@KRBTEST.COM
I20260811 22:13:03.149014 15912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.137385 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:41655 (local address 127.0.0.1:41746)
0811 22:13:03.137602 (+ 217us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:03.137619 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:03.137713 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:03.138639 (+ 926us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:03.138643 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:03.139370 (+ 727us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:03.139509 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.139519 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.140906 (+ 1387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.140911 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.141867 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.141881 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.142867 (+ 986us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.142871 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.142996 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.143522 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:03.143538 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:03.145019 (+ 1481us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:03.147202 (+ 2183us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:03.147206 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:03.147209 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:03.147447 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:03.147783 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:03.147787 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:03.147789 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:03.147884 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:03.148385 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:03.148388 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:03.148520 (+ 132us) client_negotiation.cc:769] Sending connection context
0811 22:13:03.148635 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 22:13:03.148817 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":121}
I20260811 22:13:03.149099 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.137489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41746 (local address 127.28.47.130:41655)
0811 22:13:03.138003 (+ 514us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:03.138023 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:03.138046 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:13:03.138151 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:03.138157 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:03.138456 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:03.138644 (+ 188us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:03.139681 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.140749 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.142027 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.142718 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.142812 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.145228 (+ 2416us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:03.145240 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:03.145255 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:03.145281 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:03.147053 (+ 1772us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:03.147585 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:03.147591 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:03.147592 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:03.147678 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:03.147982 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:03.147987 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:03.147988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:03.148270 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:03.148428 (+ 158us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:03.148730 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 22:13:03.148927 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":397,"thread_start_us":117,"threads_started":1}
I20260811 22:13:03.150194 15999 tablet_service.cc:1514] Processing DeleteTablet for tablet 54de7dd3659948498529679ca34b90e6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:12:42 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:41746
I20260811 22:13:03.151260 16088 tablet_replica.cc:330] stopping tablet replica
I20260811 22:13:03.151638 16088 raft_consensus.cc:2229] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:03.151953 16088 raft_consensus.cc:2258] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:03.153321 16088 ts_tablet_manager.cc:1905] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:03.158521 16088 ts_tablet_manager.cc:1918] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:03.158663 16088 log.cc:1196] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/54de7dd3659948498529679ca34b90e6
I20260811 22:13:03.158977 16088 ts_tablet_manager.cc:1939] T 54de7dd3659948498529679ca34b90e6 P 83417c0675ce4cb6a37e318208804c87: Deleting consensus metadata
I20260811 22:13:03.161264 15650 catalog_manager.cc:4841] TS 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655): tablet 54de7dd3659948498529679ca34b90e6 (table test-table [id=00c6577b87ec4b83836f277a42812deb]) successfully deleted
W20260811 22:13:03.251072 16084 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:03.251405 16084 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:03.251484 16084 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:03.255757 16084 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:03.255862 16084 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:03.255961 16084 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:13:03.261792 16084 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:46445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=36687
--tserver_master_addrs=127.28.47.190:38587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16084
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:03.263080 16084 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:03.264253 16084 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:03.275188 16094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:03.275188 16098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:03.275367 16084 server_base.cc:1034] running on GCE node
W20260811 22:13:03.275648 16095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:03.276146 16084 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:03.276926 16084 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:03.278141 16084 hybrid_clock.cc:648] HybridClock initialized: now 1786486383278104 us; error 48 us; skew 500 ppm
Aug 11 22:13:03 dist-test-slave-vxj2 krb5kdc[14946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486383, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:03.281224 16084 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:13:03.282308 16084 webserver.cc:460] Webserver started at http://127.28.47.131:36687/ using document root <none> and password file <none>
I20260811 22:13:03.282989 16084 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:03.283111 16084 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:03.287511 16084 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:13:03.290174 16106 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:03.291798 16084 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:03.291965 16084 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "8c1255f65a354397a1183c45130a455c"
format_stamp: "Formatted at 2026-08-11 22:12:42 on dist-test-slave-vxj2"
server_key: "4d321f1fa62ca027b21aaeacbc47e81e"
server_key_iv: "f294d142abf3a65c66482b03dad9180a"
server_key_version: "encryptionkey@0"
I20260811 22:13:03.292639 16084 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:03.310719 16084 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:03.314384 16084 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:03.314646 16084 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:03.315362 16084 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:03.316349 16084 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:03.316435 16084 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:03.316516 16084 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:03.316567 16084 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:03.328121 16084 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:46445
I20260811 22:13:03.328154 16250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:46445 every 8 connection(s)
I20260811 22:13:03.329445 16084 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:13:03.336553 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 16084
Aug 11 22:13:03 dist-test-slave-vxj2 krb5kdc[14946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486383, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:13:03.346146 15907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.331705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:42739 (local address 127.28.47.190:38587)
0811 22:13:03.331995 (+ 290us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:03.331999 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:03.332879 (+ 880us) server_negotiation.cc:407] Connection header received
0811 22:13:03.334270 (+ 1391us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:03.334273 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:03.334325 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:03.334404 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:03.335704 (+ 1300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.336294 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.338106 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.338511 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.338639 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.341809 (+ 3170us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:03.341812 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:03.341814 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:03.341834 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:03.343506 (+ 1672us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:03.344242 (+ 736us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:03.344245 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:03.344246 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:03.344332 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:03.344777 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:03.344779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:03.344781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:03.344971 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:03.345109 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:03.345751 (+ 642us) server_negotiation.cc:299] Negotiation successful
0811 22:13:03.345961 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178}
I20260811 22:13:03.346547 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.337482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45490 (local address 127.28.47.190:38587)
0811 22:13:03.337767 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:03.337772 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:03.337950 (+ 178us) server_negotiation.cc:407] Connection header received
0811 22:13:03.338036 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:03.338041 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:03.338111 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:03.338183 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:03.339153 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.339928 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.341210 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.341467 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.341562 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.343056 (+ 1494us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:03.343059 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:03.343061 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:03.343100 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:03.344800 (+ 1700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:03.345262 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:03.345267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:03.345269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:03.345354 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:03.345664 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:03.345668 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:03.345669 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:03.345847 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:03.346086 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:03.346230 (+ 144us) server_negotiation.cc:299] Negotiation successful
0811 22:13:03.346353 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":90,"threads_started":1}
I20260811 22:13:03.347632 16253 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.332081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:42739)
0811 22:13:03.332630 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:03.332677 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:03.334037 (+ 1360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:03.334580 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:03.334587 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:03.335029 (+ 442us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:03.335547 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.335561 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.336439 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.336445 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.337959 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.337973 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.338709 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.338713 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.338938 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.339623 (+ 685us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:03.339645 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:03.341621 (+ 1976us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:03.343653 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:03.343662 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:03.343685 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:03.344121 (+ 436us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:03.344434 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:03.344437 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:03.344439 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:03.344653 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:03.345109 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:03.345117 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:03.345466 (+ 349us) client_negotiation.cc:769] Sending connection context
0811 22:13:03.345682 (+ 216us) client_negotiation.cc:240] Negotiation successful
0811 22:13:03.346080 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":135,"threads_started":1}
I20260811 22:13:03.349007 16251 heartbeater.cc:347] Connected to a master server at 127.28.47.190:38587
I20260811 22:13:03.349323 16251 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:03.349900 16251 heartbeater.cc:511] Master 127.28.47.190:38587 requested a full tablet report, sending...
I20260811 22:13:03.351068 15665 ts_manager.cc:194] Registered new tserver with Master: 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445)
I20260811 22:13:03.351696 15665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:42739
I20260811 22:13:03.360747 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:13:03.365571 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.361946 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45504 (local address 127.28.47.190:38587)
0811 22:13:03.362106 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:03.362109 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:03.362354 (+ 245us) server_negotiation.cc:407] Connection header received
0811 22:13:03.362517 (+ 163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:03.362520 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:03.362640 (+ 120us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:03.362736 (+ 96us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:13:03.363031 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.363559 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.364376 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.364605 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.364682 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.365028 (+ 346us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:13:03.365206 (+ 178us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0811 22:13:03.365452 (+ 246us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:45504: token expired
Metrics: {"server-negotiator.queue_time_us":67}
W20260811 22:13:03.365681 16255 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:45504: token expired
W20260811 22:13:03.366168 15578 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.28.47.190:38587)
I20260811 22:13:03.373939 16255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.366622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45516 (local address 127.28.47.190:38587)
0811 22:13:03.366751 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:03.366754 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:03.366766 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:13:03.366867 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:03.366869 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:03.366910 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:03.366980 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:03.367623 (+ 643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.368127 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.368916 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.369139 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.369288 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.370639 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:03.370643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:03.370645 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:03.370678 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:03.372139 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:03.372583 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:03.372586 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:03.372587 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:03.372669 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:03.372934 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:03.372936 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:03.372937 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:03.373129 (+ 192us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:03.373333 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:03.373621 (+ 288us) server_negotiation.cc:299] Negotiation successful
0811 22:13:03.373751 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:13:03.377213 15665 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45516:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:13:03.381194 15665 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:13:03.392527 15999 tablet_service.cc:1467] Processing CreateTablet for tablet 5f1923bf499f444a919168e278723168 (DEFAULT_TABLE table=test-table [id=2d8b0e7e605b4d48b4975901d770774d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:03.392527 15809 tablet_service.cc:1467] Processing CreateTablet for tablet 5f1923bf499f444a919168e278723168 (DEFAULT_TABLE table=test-table [id=2d8b0e7e605b4d48b4975901d770774d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:03.393287 15999 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f1923bf499f444a919168e278723168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:03.393296 15809 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f1923bf499f444a919168e278723168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:03.396689 15761 tablet_bootstrap.cc:492] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4: Bootstrap starting.
I20260811 22:13:03.397348 15939 tablet_bootstrap.cc:492] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87: Bootstrap starting.
I20260811 22:13:03.397955 15761 tablet_bootstrap.cc:654] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:03.398262 15912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.391668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:46445 (local address 127.0.0.1:58202)
0811 22:13:03.391816 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:03.391836 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:03.391946 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:03.393336 (+ 1390us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:03.393340 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:03.393366 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:03.393505 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.393515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.395024 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.395030 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.396021 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.396035 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.397672 (+ 1637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.397677 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.397812 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.397909 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 22:13:03.397984 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:13:03.398091 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 22:13:03.398289 15939 tablet_bootstrap.cc:654] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:03.399292 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.392126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58202 (local address 127.28.47.131:46445)
0811 22:13:03.392696 (+ 570us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:03.392708 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:03.392760 (+ 52us) server_negotiation.cc:407] Connection header received
0811 22:13:03.392858 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:03.392865 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:03.393110 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:03.393358 (+ 248us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:03.393627 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.394897 (+ 1270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.396177 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.397311 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.398651 (+ 1340us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.398871 (+ 220us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:03.398950 (+ 79us) server_negotiation.cc:299] Negotiation successful
0811 22:13:03.399087 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":449,"thread_start_us":214,"threads_started":1}
I20260811 22:13:03.399778 15761 tablet_bootstrap.cc:492] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4: No bootstrap required, opened a new log
I20260811 22:13:03.399860 15761 ts_tablet_manager.cc:1397] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260811 22:13:03.400476 15761 raft_consensus.cc:348] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:03.400637 15761 raft_consensus.cc:374] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:03.400691 15761 raft_consensus.cc:729] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fee780109c4d4afd81a3e7319de95ef4, State: Initialized, Role: FOLLOWER
I20260811 22:13:03.400853 15761 consensus_queue.cc:260] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:03.401235 15761 ts_tablet_manager.cc:1428] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260811 22:13:03.402125 15939 tablet_bootstrap.cc:492] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87: No bootstrap required, opened a new log
I20260811 22:13:03.402240 15939 ts_tablet_manager.cc:1397] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260811 22:13:03.402650 15939 raft_consensus.cc:348] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:03.402745 15939 raft_consensus.cc:374] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:03.402776 15939 raft_consensus.cc:729] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 83417c0675ce4cb6a37e318208804c87, State: Initialized, Role: FOLLOWER
I20260811 22:13:03.402947 15939 consensus_queue.cc:260] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:03.403259 15939 ts_tablet_manager.cc:1428] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260811 22:13:03.403093 16163 tablet_service.cc:1467] Processing CreateTablet for tablet 5f1923bf499f444a919168e278723168 (DEFAULT_TABLE table=test-table [id=2d8b0e7e605b4d48b4975901d770774d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:03.404372 16163 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5f1923bf499f444a919168e278723168. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:03.412907 16266 tablet_bootstrap.cc:492] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c: Bootstrap starting.
I20260811 22:13:03.415577 16266 tablet_bootstrap.cc:654] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:03.416340 16266 log.cc:824] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:03.418669 16266 tablet_bootstrap.cc:492] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c: No bootstrap required, opened a new log
I20260811 22:13:03.418859 16266 ts_tablet_manager.cc:1397] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:13:03.422048 16266 raft_consensus.cc:348] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:03.422299 16266 raft_consensus.cc:374] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:03.422336 16266 raft_consensus.cc:729] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8c1255f65a354397a1183c45130a455c, State: Initialized, Role: FOLLOWER
I20260811 22:13:03.422766 16266 consensus_queue.cc:260] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:03.423605 16251 heartbeater.cc:503] Master 127.28.47.190:38587 was elected leader, sending a full tablet report...
I20260811 22:13:03.423933 16266 ts_tablet_manager.cc:1428] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:13:03.530339 16085 raft_consensus.cc:482] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:13:03.530691 16085 raft_consensus.cc:504] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:03.532115 16085 leader_election.cc:288] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445), fee780109c4d4afd81a3e7319de95ef4 (127.28.47.129:40159)
I20260811 22:13:03.536430 16270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.532568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:37633 (local address 127.28.47.129:40159)
0811 22:13:03.532908 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:03.532922 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:03.533093 (+ 171us) server_negotiation.cc:407] Connection header received
0811 22:13:03.533164 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:03.533167 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:03.533251 (+ 84us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:03.533375 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:03.533685 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.534298 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.535372 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.535859 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.535986 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.536158 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:03.536235 (+ 77us) server_negotiation.cc:299] Negotiation successful
0811 22:13:03.536312 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":115,"threads_started":1}
I20260811 22:13:03.536422 16082 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.532367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:46445 (local address 127.28.47.130:58695)
0811 22:13:03.532606 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:03.532624 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:03.532750 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:03.533176 (+ 426us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:03.533179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:03.533203 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:03.533361 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.533372 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.534190 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.534194 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.535173 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.535184 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.535877 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.535882 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.536001 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.536083 (+ 82us) client_negotiation.cc:769] Sending connection context
0811 22:13:03.536162 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:13:03.536253 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":112}
I20260811 22:13:03.536430 16263 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.532524 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:58695 (local address 127.28.47.131:46445)
0811 22:13:03.532667 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:03.532673 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:03.532692 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:13:03.532935 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:03.532939 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:03.533010 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:03.533097 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:03.533489 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.534067 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.535308 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.535741 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.535840 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.535885 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:03.536205 (+ 320us) server_negotiation.cc:299] Negotiation successful
0811 22:13:03.536282 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:13:03.536444 16269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.532459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:40159 (local address 127.28.47.130:37633)
0811 22:13:03.532898 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:03.532940 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:03.533051 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:03.533393 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:03.533395 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:03.533412 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:03.533560 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.533567 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.534417 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.534420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.535214 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:03.535223 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.536011 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.536014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:03.536102 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.536119 (+ 17us) client_negotiation.cc:769] Sending connection context
0811 22:13:03.536208 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:13:03.536281 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":2,"thread_start_us":130,"threads_started":1}
I20260811 22:13:03.537269 16187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1923bf499f444a919168e278723168" candidate_uuid: "83417c0675ce4cb6a37e318208804c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1255f65a354397a1183c45130a455c" is_pre_election: true
I20260811 22:13:03.537261 15832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1923bf499f444a919168e278723168" candidate_uuid: "83417c0675ce4cb6a37e318208804c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fee780109c4d4afd81a3e7319de95ef4" is_pre_election: true
I20260811 22:13:03.537631 16187 raft_consensus.cc:2454] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83417c0675ce4cb6a37e318208804c87 in term 0.
I20260811 22:13:03.537662 15832 raft_consensus.cc:2454] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 83417c0675ce4cb6a37e318208804c87 in term 0.
I20260811 22:13:03.538223 15936 leader_election.cc:302] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83417c0675ce4cb6a37e318208804c87, 8c1255f65a354397a1183c45130a455c; no voters:
I20260811 22:13:03.538542 16085 raft_consensus.cc:2781] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:13:03.538635 16085 raft_consensus.cc:482] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:13:03.538679 16085 raft_consensus.cc:3037] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:03.539788 16085 raft_consensus.cc:504] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:03.540217 16085 leader_election.cc:288] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [CANDIDATE]: Term 1 election: Requested vote from peers 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445), fee780109c4d4afd81a3e7319de95ef4 (127.28.47.129:40159)
I20260811 22:13:03.540580 16187 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1923bf499f444a919168e278723168" candidate_uuid: "83417c0675ce4cb6a37e318208804c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8c1255f65a354397a1183c45130a455c"
I20260811 22:13:03.540740 16187 raft_consensus.cc:3037] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:03.540776 15832 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5f1923bf499f444a919168e278723168" candidate_uuid: "83417c0675ce4cb6a37e318208804c87" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fee780109c4d4afd81a3e7319de95ef4"
I20260811 22:13:03.540974 15832 raft_consensus.cc:3037] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:03.542021 16187 raft_consensus.cc:2454] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83417c0675ce4cb6a37e318208804c87 in term 1.
I20260811 22:13:03.542096 15832 raft_consensus.cc:2454] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 83417c0675ce4cb6a37e318208804c87 in term 1.
I20260811 22:13:03.542421 15936 leader_election.cc:302] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 83417c0675ce4cb6a37e318208804c87, 8c1255f65a354397a1183c45130a455c; no voters:
I20260811 22:13:03.542604 16085 raft_consensus.cc:2781] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:03.542840 16085 raft_consensus.cc:686] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [term 1 LEADER]: Becoming Leader. State: Replica: 83417c0675ce4cb6a37e318208804c87, State: Running, Role: LEADER
I20260811 22:13:03.543146 16085 consensus_queue.cc:237] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:03.546798 15658 catalog_manager.cc:5495] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 reported cstate change: term changed from 0 to 1, leader changed from <none> to 83417c0675ce4cb6a37e318208804c87 (127.28.47.130). New cstate: current_term: 1 leader_uuid: "83417c0675ce4cb6a37e318208804c87" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } health_report { overall_health: HEALTHY } } }
I20260811 22:13:03.563422 16086 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.559384 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41756 (local address 127.28.47.130:41655)
0811 22:13:03.559543 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:03.559548 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:03.559565 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:13:03.559706 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:03.559708 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:03.559779 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:03.559902 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:13:03.560165 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.560719 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.561850 (+ 1131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:03.562098 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:03.562212 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:03.562507 (+ 295us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:13:03.562993 (+ 486us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:13:03.563126 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:03.563190 (+ 64us) server_negotiation.cc:299] Negotiation successful
0811 22:13:03.563254 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:13:03.568190 16187 raft_consensus.cc:1261] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c [term 1 FOLLOWER]: Refusing update from remote peer 83417c0675ce4cb6a37e318208804c87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:03.568187 15832 raft_consensus.cc:1261] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Refusing update from remote peer 83417c0675ce4cb6a37e318208804c87: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:03.569060 16272 consensus_queue.cc:1035] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:03.569322 16085 consensus_queue.cc:1035] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [LEADER]: Connected to new peer: Peer: permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:03.579942 16281 mvcc.cc:205] Tried to move back new op lower bound from 7317448227086991360 to 7317448226995355648. Current Snapshot: MvccSnapshot[applied={T|T < 7317448227086991360}]
I20260811 22:13:03.587014 15658 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:03.587245 15658 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45516:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:03.590593 15658 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Sending DeleteTablet for 3 replicas of tablet 5f1923bf499f444a919168e278723168
I20260811 22:13:03.591575 15999 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f1923bf499f444a919168e278723168 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:03 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:41746
I20260811 22:13:03.591728 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 15709
I20260811 22:13:03.591948 16163 tablet_service.cc:1514] Processing DeleteTablet for tablet 5f1923bf499f444a919168e278723168 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:03 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:58202
I20260811 22:13:03.592305 16088 tablet_replica.cc:330] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87: stopping tablet replica
I20260811 22:13:03.592458 16088 raft_consensus.cc:2229] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:13:03.592829 16088 raft_consensus.cc:2258] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:03.595446 16088 ts_tablet_manager.cc:1905] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:03.598678 16088 ts_tablet_manager.cc:1918] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:03.598795 16088 log.cc:1196] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/5f1923bf499f444a919168e278723168
I20260811 22:13:03.599080 16088 ts_tablet_manager.cc:1939] T 5f1923bf499f444a919168e278723168 P 83417c0675ce4cb6a37e318208804c87: Deleting consensus metadata
I20260811 22:13:03.600165 15650 catalog_manager.cc:4841] TS 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655): tablet 5f1923bf499f444a919168e278723168 (table test-table [id=2d8b0e7e605b4d48b4975901d770774d]) successfully deleted
I20260811 22:13:03.601557 16291 tablet_replica.cc:330] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c: stopping tablet replica
I20260811 22:13:03.602025 16291 raft_consensus.cc:2229] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:03.602453 16291 raft_consensus.cc:2258] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:03.604219 16291 ts_tablet_manager.cc:1905] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260811 22:13:03.604961 15648 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:40159 (error 108)
I20260811 22:13:03.605753 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 15908
I20260811 22:13:03.606256 15912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:03.605875 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:40159 (local address 127.0.0.1:37818)
0811 22:13:03.606084 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:03.606162 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:40159: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125}
W20260811 22:13:03.607376 15648 catalog_manager.cc:4584] TS fee780109c4d4afd81a3e7319de95ef4 (127.28.47.129:40159): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5f1923bf499f444a919168e278723168: Network error: Client connection negotiation failed: client connection to 127.28.47.129:40159: connect: Connection refused (error 111)
I20260811 22:13:03.609799 16291 ts_tablet_manager.cc:1918] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:03.609951 16291 log.cc:1196] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/5f1923bf499f444a919168e278723168
I20260811 22:13:03.610306 16291 ts_tablet_manager.cc:1939] T 5f1923bf499f444a919168e278723168 P 8c1255f65a354397a1183c45130a455c: Deleting consensus metadata
I20260811 22:13:03.611817 15650 catalog_manager.cc:4841] TS 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445): tablet 5f1923bf499f444a919168e278723168 (table test-table [id=2d8b0e7e605b4d48b4975901d770774d]) successfully deleted
I20260811 22:13:03.616091 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 16084
I20260811 22:13:03.628541 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 15627
I20260811 22:13:23.639052 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:38587
--webserver_interface=127.28.47.190
--webserver_port=39893
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:38587
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:23.760133 16293 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:23.760452 16293 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:23.760499 16293 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:23.765043 16293 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260811 22:13:23.765132 16293 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260811 22:13:23.765169 16293 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260811 22:13:23.765185 16293 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:23.765205 16293 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:23.765224 16293 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:23.765240 16293 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:23.765317 16293 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:23.771123 16293 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:38587
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:38587
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=39893
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16293
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:23.772444 16293 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:23.773661 16293 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:23.783573 16300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:23.783648 16303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:23.783578 16301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:23.783612 16293 server_base.cc:1034] running on GCE node
I20260811 22:13:23.784624 16293 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:23.786022 16293 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:23.787223 16293 hybrid_clock.cc:648] HybridClock initialized: now 1786486403787216 us; error 37 us; skew 500 ppm
Aug 11 22:13:23 dist-test-slave-vxj2 krb5kdc[14946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486403, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:23.794806 16293 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:13:23.795855 16293 webserver.cc:460] Webserver started at http://127.28.47.190:39893/ using document root <none> and password file <none>
I20260811 22:13:23.796489 16293 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:23.796593 16293 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:23.801054 16293 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:13:23.803548 16313 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:23.805088 16293 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:13:23.805335 16293 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "47bf86976000438eaa8a65d61307a11a"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "dcf62a0f7e96e0792b4b1bb5b6f42ffe"
server_key_iv: "0a9286c919562c1f77f4912af10d6db3"
server_key_version: "encryptionkey@0"
I20260811 22:13:23.805961 16293 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:23.830438 16293 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:23.834215 16293 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:23.834515 16293 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:23.843247 16293 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:38587
I20260811 22:13:23.843581 16378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:38587 every 8 connection(s)
I20260811 22:13:23.844568 16293 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:23.848555 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 16293
I20260811 22:13:23.849104 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:40159
--local_ip_for_outbound_sockets=127.28.47.129
--tserver_master_addrs=127.28.47.190:38587
--webserver_port=42857
--webserver_interface=127.28.47.129
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:13:23.849995 16380 sys_catalog.cc:263] Verifying existing consensus state
I20260811 22:13:23.852443 16380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Bootstrap starting.
I20260811 22:13:23.872504 16380 log.cc:824] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:23.884687 16380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:13:23.885280 16380 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Bootstrap complete.
I20260811 22:13:23.889632 16380 raft_consensus.cc:348] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } }
I20260811 22:13:23.890115 16380 raft_consensus.cc:729] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 47bf86976000438eaa8a65d61307a11a, State: Initialized, Role: FOLLOWER
I20260811 22:13:23.890694 16380 consensus_queue.cc:260] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } }
I20260811 22:13:23.890857 16380 raft_consensus.cc:388] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:23.890960 16380 raft_consensus.cc:482] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:23.891085 16380 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 2 FOLLOWER]: Advancing to term 3
I20260811 22:13:23.893335 16380 raft_consensus.cc:504] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } }
I20260811 22:13:23.893711 16380 leader_election.cc:302] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 47bf86976000438eaa8a65d61307a11a; no voters:
I20260811 22:13:23.894074 16380 leader_election.cc:288] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [CANDIDATE]: Term 3 election: Requested vote from peers
I20260811 22:13:23.894235 16385 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 3 FOLLOWER]: Leader election won for term 3
I20260811 22:13:23.894526 16385 raft_consensus.cc:686] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [term 3 LEADER]: Becoming Leader. State: Replica: 47bf86976000438eaa8a65d61307a11a, State: Running, Role: LEADER
I20260811 22:13:23.894936 16385 consensus_queue.cc:237] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } }
I20260811 22:13:23.895443 16380 sys_catalog.cc:564] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:23.897306 16387 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 47bf86976000438eaa8a65d61307a11a. Latest consensus state: current_term: 3 leader_uuid: "47bf86976000438eaa8a65d61307a11a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } } }
I20260811 22:13:23.897450 16387 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:23.897387 16386 sys_catalog.cc:455] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "47bf86976000438eaa8a65d61307a11a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "47bf86976000438eaa8a65d61307a11a" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 38587 } } }
I20260811 22:13:23.897507 16386 sys_catalog.cc:458] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:23.898162 16396 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:23.903816 16396 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:23.904559 16396 catalog_manager.cc:1259] Loaded cluster ID: 75c130037f694db28c9841f1a71bacba
I20260811 22:13:23.904635 16396 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:23.907794 16396 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:23.908665 16396 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Loaded TSK: 0
I20260811 22:13:23.908779 16396 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 47bf86976000438eaa8a65d61307a11a: Loaded TSK: 1
F20260811 22:13:23.909586 16396 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f6ac31409ad google::LogMessage::Fail() at ??:0
@ 0x7f6ac314388a google::LogMessage::SendToLog() at ??:0
@ 0x7f6ac31404de google::LogMessage::Flush() at ??:0
@ 0x7f6ac3141a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f6ac5e0418d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f6ac5dfcf0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f6acdaa57bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f6acda7f99c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f6acda80178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f6acdac1df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f6acdae082b std::function<>::operator()() at ??:0
@ 0x7f6acda7fb4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f6acda80f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f6acda7d579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f6acdac1640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6ac53923d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f6ac5392c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f6ac53943f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f6ac5383512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f6ac69c76db start_thread at ??:0
@ 0x7f6ac1c3e61f clone at ??:0
W20260811 22:13:24.020087 16383 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:24.020407 16383 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:24.020455 16383 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:24.024911 16383 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:24.024995 16383 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:24.025060 16383 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:13:24.030957 16383 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:40159
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=42857
--tserver_master_addrs=127.28.47.190:38587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16383
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:24.032315 16383 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:24.033557 16383 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:24.044229 16438 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:24.044288 16436 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:24.044963 16440 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:24.044960 16383 server_base.cc:1034] running on GCE node
I20260811 22:13:24.045547 16383 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:24.046409 16383 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:24.048087 16383 hybrid_clock.cc:648] HybridClock initialized: now 1786486404047971 us; error 137 us; skew 500 ppm
Aug 11 22:13:24 dist-test-slave-vxj2 krb5kdc[14946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:24.052500 16383 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:13:24.053922 16383 webserver.cc:460] Webserver started at http://127.28.47.129:42857/ using document root <none> and password file <none>
I20260811 22:13:24.054620 16383 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:24.054731 16383 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:24.060194 16383 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:13:24.063655 16450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:24.065726 16383 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:13:24.065989 16383 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "fee780109c4d4afd81a3e7319de95ef4"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "1bb06e61ecebde41310686ebbb0e61e5"
server_key_iv: "6176e3b70680eea1f8d1ec2091d2da30"
server_key_version: "encryptionkey@0"
I20260811 22:13:24.066869 16383 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:24.086658 16383 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:24.091367 16383 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:24.091609 16383 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:24.092593 16383 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:24.094254 16457 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260811 22:13:24.099316 16383 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260811 22:13:24.099426 16383 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260811 22:13:24.099515 16383 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260811 22:13:24.102138 16383 ts_tablet_manager.cc:610] Registered 1 tablets
I20260811 22:13:24.102218 16383 ts_tablet_manager.cc:589] Time spent register tablets: real 0.003s user 0.002s sys 0.000s
I20260811 22:13:24.102643 16457 tablet_bootstrap.cc:492] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4: Bootstrap starting.
I20260811 22:13:24.114900 16383 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:40159
I20260811 22:13:24.115283 16585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:40159 every 8 connection(s)
I20260811 22:13:24.116490 16383 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:13:24.119197 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 16383
I20260811 22:13:24.119765 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:41655
--local_ip_for_outbound_sockets=127.28.47.130
--tserver_master_addrs=127.28.47.190:38587
--webserver_port=42601
--webserver_interface=127.28.47.130
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:13:24.125881 16457 log.cc:824] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:24.133601 16457 tablet_bootstrap.cc:492] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260811 22:13:24.134063 16457 tablet_bootstrap.cc:492] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4: Bootstrap complete.
I20260811 22:13:24.135290 16457 ts_tablet_manager.cc:1397] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4: Time spent bootstrapping tablet: real 0.033s user 0.026s sys 0.004s
I20260811 22:13:24.138098 16457 raft_consensus.cc:348] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:24.138583 16457 raft_consensus.cc:729] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: fee780109c4d4afd81a3e7319de95ef4, State: Initialized, Role: FOLLOWER
I20260811 22:13:24.139117 16457 consensus_queue.cc:260] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:24.140161 16457 ts_tablet_manager.cc:1428] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
W20260811 22:13:24.195833 16588 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:13:24.119278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:53159)
0811 22:13:24.120079 (+ 801us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:24.120148 (+ 69us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:24.121321 (+ 1173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:24.194785 (+ 73464us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":537,"thread_start_us":336,"threads_started":1}
W20260811 22:13:24.196519 16586 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:38587 (0 consecutive failures): Network error: Failed to ping master at 127.28.47.190:38587: Client connection negotiation failed: client connection to 127.28.47.190:38587: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:13:24.197474 16588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:24.197117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:40227)
0811 22:13:24.197306 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:24.197397 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":88}
I20260811 22:13:24.198362 16588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:24.198116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:51149)
0811 22:13:24.198242 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:24.198297 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260811 22:13:24.198599 16586 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:13:24.262328 16589 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:24.262634 16589 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:24.262724 16589 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:24.267017 16589 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:24.267095 16589 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:24.267201 16589 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:13:24.273000 16589 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:41655
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=42601
--tserver_master_addrs=127.28.47.190:38587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16589
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:24.274328 16589 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:24.275532 16589 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:24.286293 16602 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:24.286291 16599 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:24.286290 16598 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:24.286603 16589 server_base.cc:1034] running on GCE node
I20260811 22:13:24.287163 16589 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:24.287927 16589 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:24.289125 16589 hybrid_clock.cc:648] HybridClock initialized: now 1786486404289083 us; error 55 us; skew 500 ppm
Aug 11 22:13:24 dist-test-slave-vxj2 krb5kdc[14946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:24.292232 16589 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:13:24.293345 16589 webserver.cc:460] Webserver started at http://127.28.47.130:42601/ using document root <none> and password file <none>
I20260811 22:13:24.293933 16589 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:24.293998 16589 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:24.298272 16589 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:13:24.300690 16609 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:24.302227 16589 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:13:24.302348 16589 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "83417c0675ce4cb6a37e318208804c87"
format_stamp: "Formatted at 2026-08-11 22:12:41 on dist-test-slave-vxj2"
server_key: "7d3b798a753038e62293be1152e76be4"
server_key_iv: "86d598879461a163634fc270f4ed67bb"
server_key_version: "encryptionkey@0"
I20260811 22:13:24.302994 16589 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:24.321854 16589 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:24.325735 16589 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:24.326004 16589 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:24.326719 16589 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:24.327698 16589 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:24.327839 16589 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:24.327922 16589 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:24.327972 16589 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:24.338130 16589 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:41655
I20260811 22:13:24.338236 16749 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:41655 every 8 connection(s)
I20260811 22:13:24.339320 16589 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:13:24.341925 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 16589
I20260811 22:13:24.342476 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:46445
--local_ip_for_outbound_sockets=127.28.47.131
--tserver_master_addrs=127.28.47.190:38587
--webserver_port=36687
--webserver_interface=127.28.47.131
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:39939
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:13:24.344484 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:24.341965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:38129)
0811 22:13:24.342657 (+ 692us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:24.343372 (+ 715us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":168,"threads_started":1}
W20260811 22:13:24.345021 16750 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:38587 (0 consecutive failures): Network error: Failed to ping master at 127.28.47.190:38587: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
I20260811 22:13:24.345830 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:24.345542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:52361)
0811 22:13:24.345673 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:24.345724 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260811 22:13:24.346563 16752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:24.346341 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:53219)
0811 22:13:24.346466 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:24.346509 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:13:24.346773 16750 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260811 22:13:24.467970 16754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:24.468370 16754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:24.468425 16754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:24.472958 16754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:24.473079 16754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:24.473218 16754 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:13:24.479142 16754 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:39939
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:46445
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=36687
--tserver_master_addrs=127.28.47.190:38587
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:24.480396 16754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:24.481586 16754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:24.493048 16762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:24.493139 16764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:24.493048 16761 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:24.493510 16754 server_base.cc:1034] running on GCE node
I20260811 22:13:24.494032 16754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:24.494889 16754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:24.496101 16754 hybrid_clock.cc:648] HybridClock initialized: now 1786486404496084 us; error 38 us; skew 500 ppm
Aug 11 22:13:24 dist-test-slave-vxj2 krb5kdc[14946](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:24.499251 16754 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:13:24.500321 16754 webserver.cc:460] Webserver started at http://127.28.47.131:36687/ using document root <none> and password file <none>
I20260811 22:13:24.500937 16754 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:24.501004 16754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:24.505574 16754 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:24.508342 16771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:24.509850 16754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:24.509986 16754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "8c1255f65a354397a1183c45130a455c"
format_stamp: "Formatted at 2026-08-11 22:12:42 on dist-test-slave-vxj2"
server_key: "4d321f1fa62ca027b21aaeacbc47e81e"
server_key_iv: "f294d142abf3a65c66482b03dad9180a"
server_key_version: "encryptionkey@0"
I20260811 22:13:24.510594 16754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:24.532544 16754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:24.536188 16754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:24.536376 16754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:24.537009 16754 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:24.537951 16754 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:24.538009 16754 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:24.538054 16754 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:24.538070 16754 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:24.548585 16754 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:46445
I20260811 22:13:24.548707 16909 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:46445 every 8 connection(s)
I20260811 22:13:24.549782 16754 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:13:24.551976 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 16754
I20260811 22:13:24.554352 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:24.552355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:48467)
0811 22:13:24.552927 (+ 572us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:24.553543 (+ 616us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":162,"threads_started":1}
W20260811 22:13:24.554912 16911 heartbeater.cc:644] Failed to heartbeat to 127.28.47.190:38587 (0 consecutive failures): Network error: Failed to ping master at 127.28.47.190:38587: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
I20260811 22:13:24.555675 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:24.555372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:55351)
0811 22:13:24.555526 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:24.555599 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 22:13:24.556528 16913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:24.556288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:48259)
0811 22:13:24.556405 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:24.556461 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 22:13:24.556864 16911 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260811 22:13:25.200047 16915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:25.199456 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:45789)
0811 22:13:25.199828 (+ 372us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:25.199915 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":84,"threads_started":1}
I20260811 22:13:25.348127 16916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:25.347612 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:36699)
0811 22:13:25.347950 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:25.348010 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":105,"threads_started":1}
I20260811 22:13:25.558158 16917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:25.557681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:48579)
0811 22:13:25.557971 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:25.558037 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":81,"threads_started":1}
I20260811 22:13:25.782361 16918 raft_consensus.cc:482] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:13:25.782665 16918 raft_consensus.cc:504] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:25.783890 16918 leader_election.cc:288] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445), 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655)
Aug 11 22:13:25 dist-test-slave-vxj2 krb5kdc[14946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.131@KRBTEST.COM
Aug 11 22:13:25 dist-test-slave-vxj2 krb5kdc[14946](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486404, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.130@KRBTEST.COM
I20260811 22:13:25.803428 16919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:25.784235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:46445 (local address 127.28.47.129:50983)
0811 22:13:25.784915 (+ 680us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:25.784930 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:25.785053 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:25.787345 (+ 2292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:25.787352 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:25.788057 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:25.788964 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:25.788989 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:25.791099 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:25.791103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:25.792885 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:25.792901 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:25.794115 (+ 1214us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:25.794119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:25.794540 (+ 421us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:25.795475 (+ 935us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:25.795509 (+ 34us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:25.797601 (+ 2092us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:25.800338 (+ 2737us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:25.800349 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:25.800367 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:25.800821 (+ 454us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:25.801359 (+ 538us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:25.801363 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:25.801367 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:25.801548 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:25.802329 (+ 781us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:25.802336 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:25.802598 (+ 262us) client_negotiation.cc:769] Sending connection context
0811 22:13:25.802775 (+ 177us) client_negotiation.cc:240] Negotiation successful
0811 22:13:25.803166 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":594,"mutex_wait_us":387,"spinlock_wait_cycles":30336,"thread_start_us":108,"threads_started":1}
I20260811 22:13:25.803494 16922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:25.785266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:33237 (local address 127.28.47.130:41655)
0811 22:13:25.785713 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:25.785737 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:25.785776 (+ 39us) server_negotiation.cc:407] Connection header received
0811 22:13:25.786679 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:25.786688 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:25.787021 (+ 333us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:25.787284 (+ 263us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:25.789233 (+ 1949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:25.790891 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:25.793083 (+ 2192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:25.793939 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:25.794146 (+ 207us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:25.798190 (+ 4044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:25.798195 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:25.798216 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:25.798261 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:25.800308 (+ 2047us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:25.801070 (+ 762us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:25.801076 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:25.801080 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:25.801312 (+ 232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:25.801655 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:25.801659 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:25.801661 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:25.802189 (+ 528us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:25.802338 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:25.803031 (+ 693us) server_negotiation.cc:299] Negotiation successful
0811 22:13:25.803319 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":112,"threads_started":1}
I20260811 22:13:25.803484 16921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:25.784772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:41655 (local address 127.28.47.129:33237)
0811 22:13:25.785319 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:25.785331 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:25.785455 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:25.787345 (+ 1890us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:25.787352 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:25.788057 (+ 705us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:25.788964 (+ 907us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:25.788989 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:25.791099 (+ 2110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:25.791103 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:25.792885 (+ 1782us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:25.792907 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:25.794115 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:25.794119 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:25.794538 (+ 419us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:25.795525 (+ 987us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:25.795539 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:25.797935 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:25.800443 (+ 2508us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:25.800446 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:25.800447 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:25.800937 (+ 490us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:25.801417 (+ 480us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:25.801419 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:25.801421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:25.801547 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:25.802329 (+ 782us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:25.802336 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:25.802658 (+ 322us) client_negotiation.cc:769] Sending connection context
0811 22:13:25.802881 (+ 223us) client_negotiation.cc:240] Negotiation successful
0811 22:13:25.803279 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":488,"mutex_wait_us":656,"spinlock_wait_cycles":6016,"thread_start_us":75,"threads_started":1}
I20260811 22:13:25.803915 16920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:25.784371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:50983 (local address 127.28.47.131:46445)
0811 22:13:25.784912 (+ 541us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:25.784967 (+ 55us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:25.785024 (+ 57us) server_negotiation.cc:407] Connection header received
0811 22:13:25.786307 (+ 1283us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:25.786318 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:25.786862 (+ 544us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:25.787245 (+ 383us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:25.789234 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:25.790890 (+ 1656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:25.793083 (+ 2193us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:25.793955 (+ 872us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:25.794146 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:25.797938 (+ 3792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:25.797946 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:25.797969 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:25.798019 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:25.800149 (+ 2130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:25.800978 (+ 829us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:25.800987 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:25.800994 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:25.801221 (+ 227us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:25.801660 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:25.801664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:25.801666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:25.802189 (+ 523us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:25.802348 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:25.802976 (+ 628us) server_negotiation.cc:299] Negotiation successful
0811 22:13:25.803342 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":398,"thread_start_us":88,"threads_started":1}
W20260811 22:13:25.806537 16454 leader_election.cc:341] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655): Not found: Tablet not found: 5f1923bf499f444a919168e278723168
W20260811 22:13:25.806879 16454 leader_election.cc:341] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445): Not found: Tablet not found: 5f1923bf499f444a919168e278723168
I20260811 22:13:25.806994 16454 leader_election.cc:302] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fee780109c4d4afd81a3e7319de95ef4; no voters: 83417c0675ce4cb6a37e318208804c87, 8c1255f65a354397a1183c45130a455c
I20260811 22:13:25.807343 16918 raft_consensus.cc:2726] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:13:26.201398 16921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:26.201022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:55111)
0811 22:13:26.201231 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:26.201300 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94}
I20260811 22:13:26.349560 16923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:26.349029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:47969)
0811 22:13:26.349391 (+ 362us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:26.349456 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":80,"threads_started":1}
I20260811 22:13:26.559526 16924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:26.559065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:34353)
0811 22:13:26.559359 (+ 294us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:26.559424 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":88,"threads_started":1}
I20260811 22:13:27.203068 16925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:27.202424 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:58269)
0811 22:13:27.202850 (+ 426us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:27.202931 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":100,"threads_started":1}
I20260811 22:13:27.351052 16926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:27.350499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:44373)
0811 22:13:27.350863 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:27.350928 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":85,"threads_started":1}
I20260811 22:13:27.561084 16927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:27.560548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:57063)
0811 22:13:27.560889 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:27.560960 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":90,"threads_started":1}
I20260811 22:13:27.967947 16928 raft_consensus.cc:482] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:13:27.968097 16928 raft_consensus.cc:504] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:27.968523 16928 leader_election.cc:288] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445), 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655)
W20260811 22:13:27.969692 16454 leader_election.cc:341] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655): Not found: Tablet not found: 5f1923bf499f444a919168e278723168
W20260811 22:13:27.969835 16454 leader_election.cc:341] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445): Not found: Tablet not found: 5f1923bf499f444a919168e278723168
I20260811 22:13:27.969945 16454 leader_election.cc:302] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fee780109c4d4afd81a3e7319de95ef4; no voters: 83417c0675ce4cb6a37e318208804c87, 8c1255f65a354397a1183c45130a455c
I20260811 22:13:27.970165 16928 raft_consensus.cc:2726] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:13:28.204571 16929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:28.204057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:35693)
0811 22:13:28.204406 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:28.204465 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":146,"threads_started":1}
I20260811 22:13:28.352512 16930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:28.352034 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:48381)
0811 22:13:28.352337 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:28.352408 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":102,"threads_started":1}
I20260811 22:13:28.562469 16931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:28.562025 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:43579)
0811 22:13:28.562303 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:28.562371 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":84,"threads_started":1}
I20260811 22:13:29.206370 16932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:29.205747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:42759)
0811 22:13:29.206148 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:29.206223 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":309,"thread_start_us":148,"threads_started":1}
I20260811 22:13:29.354092 16933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:29.353585 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:39333)
0811 22:13:29.353928 (+ 343us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:29.353992 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":257,"thread_start_us":95,"threads_started":1}
I20260811 22:13:29.564121 16934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:29.563531 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:44103)
0811 22:13:29.563922 (+ 391us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:29.564009 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":90,"threads_started":1}
I20260811 22:13:30.208344 16935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:30.207732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:54049)
0811 22:13:30.208151 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:30.208221 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":106,"threads_started":1}
I20260811 22:13:30.355603 16936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:30.355086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:40743)
0811 22:13:30.355425 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:30.355498 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":86,"threads_started":1}
I20260811 22:13:30.565912 16937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:30.565380 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:57203)
0811 22:13:30.565699 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:30.565766 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":229,"thread_start_us":91,"threads_started":1}
I20260811 22:13:30.684253 16938 raft_consensus.cc:482] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:13:30.684409 16938 raft_consensus.cc:504] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:30.684993 16938 leader_election.cc:288] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445), 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655)
W20260811 22:13:30.686282 16454 leader_election.cc:341] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655): Not found: Tablet not found: 5f1923bf499f444a919168e278723168
W20260811 22:13:30.686496 16454 leader_election.cc:341] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445): Not found: Tablet not found: 5f1923bf499f444a919168e278723168
I20260811 22:13:30.686566 16454 leader_election.cc:302] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fee780109c4d4afd81a3e7319de95ef4; no voters: 83417c0675ce4cb6a37e318208804c87, 8c1255f65a354397a1183c45130a455c
I20260811 22:13:30.686995 16938 raft_consensus.cc:2726] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:13:31.210119 16939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:31.209441 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:38781)
0811 22:13:31.209917 (+ 476us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:31.209997 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":360,"thread_start_us":93,"threads_started":1}
I20260811 22:13:31.357091 16940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:31.356617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:59767)
0811 22:13:31.356934 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:31.356991 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":89,"threads_started":1}
I20260811 22:13:31.567481 16941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:31.566944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:57447)
0811 22:13:31.567308 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:31.567383 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":77,"threads_started":1}
I20260811 22:13:32.211498 16942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:32.211038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:40075)
0811 22:13:32.211328 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:32.211398 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":83,"threads_started":1}
I20260811 22:13:32.358541 16943 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:32.358032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:41219)
0811 22:13:32.358338 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:32.358422 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":88,"threads_started":1}
I20260811 22:13:32.569077 16944 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:32.568552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:41261)
0811 22:13:32.568864 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:32.568937 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":126,"threads_started":1}
I20260811 22:13:32.912458 16945 raft_consensus.cc:482] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:13:32.912621 16945 raft_consensus.cc:504] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8c1255f65a354397a1183c45130a455c" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46445 } } peers { permanent_uuid: "fee780109c4d4afd81a3e7319de95ef4" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 40159 } } peers { permanent_uuid: "83417c0675ce4cb6a37e318208804c87" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41655 } }
I20260811 22:13:32.913211 16945 leader_election.cc:288] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445), 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655)
W20260811 22:13:32.914201 16454 leader_election.cc:341] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 8c1255f65a354397a1183c45130a455c (127.28.47.131:46445): Not found: Tablet not found: 5f1923bf499f444a919168e278723168
W20260811 22:13:32.914384 16454 leader_election.cc:341] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 83417c0675ce4cb6a37e318208804c87 (127.28.47.130:41655): Not found: Tablet not found: 5f1923bf499f444a919168e278723168
I20260811 22:13:32.914460 16454 leader_election.cc:302] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: fee780109c4d4afd81a3e7319de95ef4; no voters: 83417c0675ce4cb6a37e318208804c87, 8c1255f65a354397a1183c45130a455c
I20260811 22:13:32.914978 16945 raft_consensus.cc:2726] T 5f1923bf499f444a919168e278723168 P fee780109c4d4afd81a3e7319de95ef4 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260811 22:13:33.212848 16946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:33.212386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:55257)
0811 22:13:33.212692 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:33.212749 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":101,"threads_started":1}
I20260811 22:13:33.359944 16947 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:33.359458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:58989)
0811 22:13:33.359769 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:33.359843 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":89,"threads_started":1}
I20260811 22:13:33.570580 16950 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:33.570050 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:34195)
0811 22:13:33.570403 (+ 353us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:33.570478 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":109,"threads_started":1}
I20260811 22:13:34.214298 16951 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:34.213843 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:59573)
0811 22:13:34.214130 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:34.214192 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":79,"threads_started":1}
I20260811 22:13:34.361490 16952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:34.360928 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:54353)
0811 22:13:34.361303 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:34.361372 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":84,"threads_started":1}
I20260811 22:13:34.572052 16955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:34.571471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:35135)
0811 22:13:34.571861 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:34.571938 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":108,"threads_started":1}
I20260811 22:13:35.215806 16956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:35.215298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:34113)
0811 22:13:35.215614 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:35.215700 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":80,"threads_started":1}
I20260811 22:13:35.363112 16957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:35.362549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:48705)
0811 22:13:35.362875 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:35.362988 (+ 113us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":115,"threads_started":1}
I20260811 22:13:35.573715 16958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:35.573195 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:38677)
0811 22:13:35.573522 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:35.573592 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":81,"threads_started":1}
I20260811 22:13:36.217309 16959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:36.216785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:44561)
0811 22:13:36.217081 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:36.217146 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":84,"threads_started":1}
I20260811 22:13:36.364671 16960 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:36.364103 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:35889)
0811 22:13:36.364467 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:36.364544 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":106,"threads_started":1}
I20260811 22:13:36.575361 16961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:36.574863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:36871)
0811 22:13:36.575189 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:36.575255 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":242,"thread_start_us":102,"threads_started":1}
I20260811 22:13:37.218731 16962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:37.218213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:58217)
0811 22:13:37.218524 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:37.218592 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":81,"threads_started":1}
I20260811 22:13:37.366024 16963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:37.365621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:54795)
0811 22:13:37.365868 (+ 247us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:37.365926 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":77,"threads_started":1}
I20260811 22:13:37.576712 16964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:37.576227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:59707)
0811 22:13:37.576548 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:37.576609 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":106,"threads_started":1}
I20260811 22:13:38.220216 16965 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:38.219720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:38543)
0811 22:13:38.220033 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:38.220100 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":90,"threads_started":1}
I20260811 22:13:38.367594 16967 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:38.367068 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:48913)
0811 22:13:38.367407 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:38.367476 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":90,"threads_started":1}
I20260811 22:13:38.578222 16968 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:38.577681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:52537)
0811 22:13:38.578039 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:38.578111 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":95,"threads_started":1}
I20260811 22:13:39.222065 16969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:39.221566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.129:39137)
0811 22:13:39.221862 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:39.221940 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":112,"threads_started":1}
I20260811 22:13:39.369493 16971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:39.368832 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.130:45059)
0811 22:13:39.369306 (+ 474us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:39.369381 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":86,"threads_started":1}
I20260811 22:13:39.561960 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 16383
I20260811 22:13:39.579342 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 16589
I20260811 22:13:39.581461 16972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:39.579389 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:38587 (local address 127.28.47.131:59215)
0811 22:13:39.581219 (+ 1830us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:39.581306 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.190:38587: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1683,"thread_start_us":81,"threads_started":1}
I20260811 22:13:39.590291 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 16754
2026-08-11T22:13:39Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (59825 ms)
[----------] 30 tests from SecurityITest (271921 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:13:39 dist-test-slave-vxj2 krb5kdc[16976](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:13:39 dist-test-slave-vxj2 krb5kdc[16976](info): set up 2 sockets
Aug 11 22:13:39 dist-test-slave-vxj2 krb5kdc[16976](info): commencing operation
krb5kdc: starting...
W20260811 22:13:40.746837 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.130s user 0.002s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:13:40 dist-test-slave-vxj2 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:13:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:40Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:13:40.886561 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:35081
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:35081
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:41.011962 16993 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:41.012326 16993 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:41.012421 16993 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:41.016696 16993 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:41.016820 16993 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:41.016861 16993 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:41.016911 16993 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:41.016944 16993 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:41.022812 16993 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:35081
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:35081
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16993
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:41.024013 16993 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:41.025275 16993 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:41.034116 17000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:41.034215 16993 server_base.cc:1034] running on GCE node
W20260811 22:13:41.034111 17001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:41.034116 17003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:41.035058 16993 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:41.036156 16993 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:41.037320 16993 hybrid_clock.cc:648] HybridClock initialized: now 1786486421037299 us; error 33 us; skew 500 ppm
Aug 11 22:13:41 dist-test-slave-vxj2 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486421, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:41.040213 16993 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:13:41.041234 16993 webserver.cc:460] Webserver started at http://127.28.47.190:40361/ using document root <none> and password file <none>
I20260811 22:13:41.041806 16993 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:41.041869 16993 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:41.042052 16993 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:41.044130 16993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "3de90885e8354b93830953f58eeb06c4"
format_stamp: "Formatted at 2026-08-11 22:13:41 on dist-test-slave-vxj2"
server_key: "bbd51e1ec80807a7cdb5d5f9e12d827d"
server_key_iv: "ef6cdf209962d6fc6a7685b6399f24e6"
server_key_version: "encryptionkey@0"
I20260811 22:13:41.044661 16993 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "3de90885e8354b93830953f58eeb06c4"
format_stamp: "Formatted at 2026-08-11 22:13:41 on dist-test-slave-vxj2"
server_key: "bbd51e1ec80807a7cdb5d5f9e12d827d"
server_key_iv: "ef6cdf209962d6fc6a7685b6399f24e6"
server_key_version: "encryptionkey@0"
I20260811 22:13:41.048792 16993 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:13:41.051627 17011 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:41.053035 16993 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:13:41.053263 16993 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "3de90885e8354b93830953f58eeb06c4"
format_stamp: "Formatted at 2026-08-11 22:13:41 on dist-test-slave-vxj2"
server_key: "bbd51e1ec80807a7cdb5d5f9e12d827d"
server_key_iv: "ef6cdf209962d6fc6a7685b6399f24e6"
server_key_version: "encryptionkey@0"
I20260811 22:13:41.053429 16993 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:41.075155 16993 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:41.079095 16993 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:41.079388 16993 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:41.088207 16993 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:35081
I20260811 22:13:41.088229 17071 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:35081 every 8 connection(s)
I20260811 22:13:41.089404 16993 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:41.092481 17072 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:41.094955 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 16993
I20260811 22:13:41.095096 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:13:41.095398 28862 external_mini_cluster.cc:1428] Setting key 91ff3434e2222d8de79fffd3cb07a857
I20260811 22:13:41.099320 17072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4: Bootstrap starting.
I20260811 22:13:41.102157 17072 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:41.103189 17072 log.cc:824] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:13:41 dist-test-slave-vxj2 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486420, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:13:41.105841 17072 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4: No bootstrap required, opened a new log
I20260811 22:13:41.109902 17072 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de90885e8354b93830953f58eeb06c4" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35081 } }
I20260811 22:13:41.110167 17072 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:41.110230 17072 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3de90885e8354b93830953f58eeb06c4, State: Initialized, Role: FOLLOWER
I20260811 22:13:41.110802 17072 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de90885e8354b93830953f58eeb06c4" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35081 } }
I20260811 22:13:41.110988 17072 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:41.111079 17072 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:41.111224 17072 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:41.112901 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.097089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34126 (local address 127.28.47.190:35081)
0811 22:13:41.097644 (+ 555us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:41.097662 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:41.097703 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:13:41.098467 (+ 764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:41.098486 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:41.098871 (+ 385us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:41.099269 (+ 398us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:41.100225 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.101577 (+ 1352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.102602 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.103093 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.103277 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.106494 (+ 3217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:41.106500 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:41.106516 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:41.106547 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:41.109235 (+ 2688us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:41.109805 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:41.109810 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:41.109816 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:41.109956 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:41.110302 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:41.110308 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:41.110310 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:41.110767 (+ 457us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:41.111041 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:41.111354 (+ 313us) server_negotiation.cc:299] Negotiation successful
0811 22:13:41.111731 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":392,"thread_start_us":174,"threads_started":1}
I20260811 22:13:41.112979 17072 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de90885e8354b93830953f58eeb06c4" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35081 } }
I20260811 22:13:41.113456 17072 leader_election.cc:302] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3de90885e8354b93830953f58eeb06c4; no voters:
I20260811 22:13:41.113901 17072 leader_election.cc:288] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:41.114017 17078 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:41.114275 17078 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [term 1 LEADER]: Becoming Leader. State: Replica: 3de90885e8354b93830953f58eeb06c4, State: Running, Role: LEADER
I20260811 22:13:41.114746 17078 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de90885e8354b93830953f58eeb06c4" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35081 } }
I20260811 22:13:41.115360 17072 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:41.116473 17079 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3de90885e8354b93830953f58eeb06c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de90885e8354b93830953f58eeb06c4" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35081 } } }
I20260811 22:13:41.116616 17079 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:41.116950 17080 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3de90885e8354b93830953f58eeb06c4. Latest consensus state: current_term: 1 leader_uuid: "3de90885e8354b93830953f58eeb06c4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3de90885e8354b93830953f58eeb06c4" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 35081 } } }
I20260811 22:13:41.117034 17087 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:41.117066 17080 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:41.120512 17087 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:41.127688 17087 catalog_manager.cc:1347] Generated new cluster ID: 7db2b7633f574c579b8cc9d3bae3e06d
I20260811 22:13:41.127799 17087 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:41.134522 17087 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:41.135603 17087 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:41.141693 17087 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4: Generated new TSK 0
I20260811 22:13:41.142542 17087 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:13:41.181363 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35081
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:41.324577 17101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:41.325286 17101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:41.325426 17101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:41.329905 17101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:41.330005 17101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:41.330102 17101 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:13:41.335826 17101 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:41.337064 17101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:41.338246 17101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:41.348672 17107 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:41.348672 17106 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:41.348696 17109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:41.349206 17101 server_base.cc:1034] running on GCE node
I20260811 22:13:41.349677 17101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:41.350366 17101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:41.351559 17101 hybrid_clock.cc:648] HybridClock initialized: now 1786486421351546 us; error 34 us; skew 500 ppm
Aug 11 22:13:41 dist-test-slave-vxj2 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486421, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:41.354591 17101 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:13:41.355557 17101 webserver.cc:460] Webserver started at http://127.28.47.129:42357/ using document root <none> and password file <none>
I20260811 22:13:41.356166 17101 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:41.356263 17101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:41.356514 17101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:41.358754 17101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "e2138d00b0e74a29ace51b72149bd3dc"
format_stamp: "Formatted at 2026-08-11 22:13:41 on dist-test-slave-vxj2"
server_key: "a510e73cd8b32cd790b274538631ef74"
server_key_iv: "6b175a44372a9340d36254fc8eb0a384"
server_key_version: "encryptionkey@0"
I20260811 22:13:41.359364 17101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "e2138d00b0e74a29ace51b72149bd3dc"
format_stamp: "Formatted at 2026-08-11 22:13:41 on dist-test-slave-vxj2"
server_key: "a510e73cd8b32cd790b274538631ef74"
server_key_iv: "6b175a44372a9340d36254fc8eb0a384"
server_key_version: "encryptionkey@0"
I20260811 22:13:41.363348 17101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:13:41.366009 17117 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:41.367264 17101 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260811 22:13:41.367413 17101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "e2138d00b0e74a29ace51b72149bd3dc"
format_stamp: "Formatted at 2026-08-11 22:13:41 on dist-test-slave-vxj2"
server_key: "a510e73cd8b32cd790b274538631ef74"
server_key_iv: "6b175a44372a9340d36254fc8eb0a384"
server_key_version: "encryptionkey@0"
I20260811 22:13:41.367540 17101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:41.386533 17101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:41.390578 17101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:41.390769 17101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:41.391408 17101 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:41.392341 17101 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:41.392391 17101 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:41.392436 17101 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:41.392452 17101 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:41.401994 17101 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:33815
I20260811 22:13:41.402061 17248 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:33815 every 8 connection(s)
I20260811 22:13:41.403072 17101 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:13:41.410918 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 17101
I20260811 22:13:41.411103 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:13:41.411453 28862 external_mini_cluster.cc:1428] Setting key 8f3acd16f29906fdba985e79ac1bc55e
Aug 11 22:13:41 dist-test-slave-vxj2 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486421, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:13:41.419334 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.405313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:43351 (local address 127.28.47.190:35081)
0811 22:13:41.405485 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:41.405492 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:41.406468 (+ 976us) server_negotiation.cc:407] Connection header received
0811 22:13:41.407830 (+ 1362us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:41.407833 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:41.407893 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:41.407976 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:41.409361 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.409917 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.411176 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.411552 (+ 376us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.411679 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.414708 (+ 3029us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:41.414714 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:41.414721 (+ 7us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:41.414758 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:41.416624 (+ 1866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:41.417427 (+ 803us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:41.417430 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:41.417431 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:41.417540 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:41.417990 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:41.417992 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:41.417993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:41.418181 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:41.418257 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:41.419074 (+ 817us) server_negotiation.cc:299] Negotiation successful
0811 22:13:41.419201 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:13:41.420737 17251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.405779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:35081 (local address 127.28.47.129:43351)
0811 22:13:41.406283 (+ 504us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:41.406330 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:41.407594 (+ 1264us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:41.408183 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:41.408190 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:41.408571 (+ 381us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:41.409109 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:41.409125 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.410048 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.410051 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:41.411004 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:41.411022 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.411725 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.411728 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:41.411943 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.412678 (+ 735us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:41.412695 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:41.414527 (+ 1832us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:41.416750 (+ 2223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:41.416759 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:41.416779 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:41.417245 (+ 466us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:41.417652 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:41.417658 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:41.417663 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:41.417883 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:41.418302 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:41.418312 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:41.418712 (+ 400us) client_negotiation.cc:769] Sending connection context
0811 22:13:41.418968 (+ 256us) client_negotiation.cc:240] Negotiation successful
0811 22:13:41.419406 (+ 438us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":124,"threads_started":1}
I20260811 22:13:41.422336 17249 heartbeater.cc:347] Connected to a master server at 127.28.47.190:35081
I20260811 22:13:41.422619 17249 heartbeater.cc:464] Registering TS with master...
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
I20260811 22:13:41.423233 17249 heartbeater.cc:511] Master 127.28.47.190:35081 requested a full tablet report, sending...
I20260811 22:13:41.424932 17028 ts_manager.cc:194] Registered new tserver with Master: e2138d00b0e74a29ace51b72149bd3dc (127.28.47.129:33815)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:13:41.426398 17028 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:43351
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:13:41.445363 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35081
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:41.564882 17256 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:41.565258 17256 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:41.565351 17256 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:41.569578 17256 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:41.569694 17256 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:41.569793 17256 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:13:41.575649 17256 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17256
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:41.576930 17256 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:41.578146 17256 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:41.588831 17261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:41.588851 17264 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:41.588887 17262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:41.589068 17256 server_base.cc:1034] running on GCE node
I20260811 22:13:41.589612 17256 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:41.590406 17256 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:41.591589 17256 hybrid_clock.cc:648] HybridClock initialized: now 1786486421591554 us; error 51 us; skew 500 ppm
Aug 11 22:13:41 dist-test-slave-vxj2 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486421, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:41.594869 17256 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:13:41.595860 17256 webserver.cc:460] Webserver started at http://127.28.47.130:38313/ using document root <none> and password file <none>
I20260811 22:13:41.596467 17256 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:41.596530 17256 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:41.596757 17256 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:41.599056 17256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "5a9108e49fe04774a83dacdc27322d16"
format_stamp: "Formatted at 2026-08-11 22:13:41 on dist-test-slave-vxj2"
server_key: "99b357d7eda4ecd54a208c77c2019fec"
server_key_iv: "0adaa79c9c6faec324dfde2af4ff7f76"
server_key_version: "encryptionkey@0"
I20260811 22:13:41.599686 17256 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "5a9108e49fe04774a83dacdc27322d16"
format_stamp: "Formatted at 2026-08-11 22:13:41 on dist-test-slave-vxj2"
server_key: "99b357d7eda4ecd54a208c77c2019fec"
server_key_iv: "0adaa79c9c6faec324dfde2af4ff7f76"
server_key_version: "encryptionkey@0"
I20260811 22:13:41.603914 17256 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.005s
I20260811 22:13:41.606848 17271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:41.608219 17256 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:13:41.608384 17256 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "5a9108e49fe04774a83dacdc27322d16"
format_stamp: "Formatted at 2026-08-11 22:13:41 on dist-test-slave-vxj2"
server_key: "99b357d7eda4ecd54a208c77c2019fec"
server_key_iv: "0adaa79c9c6faec324dfde2af4ff7f76"
server_key_version: "encryptionkey@0"
I20260811 22:13:41.608526 17256 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:41.644130 17256 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:41.647368 17256 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:41.647632 17256 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:41.648276 17256 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:41.649215 17256 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:41.649286 17256 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:41.649358 17256 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:41.649407 17256 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:41.658970 17256 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:38647
I20260811 22:13:41.658994 17401 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:38647 every 8 connection(s)
I20260811 22:13:41.659929 17256 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:13:41.663360 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 17256
I20260811 22:13:41.663525 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:13:41.663879 28862 external_mini_cluster.cc:1428] Setting key b3997dfdc78ec6ff600aa65de82bb5c6
Aug 11 22:13:41 dist-test-slave-vxj2 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486421, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:13:41.675659 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.661975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:42179 (local address 127.28.47.190:35081)
0811 22:13:41.662115 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:41.662118 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:41.663065 (+ 947us) server_negotiation.cc:407] Connection header received
0811 22:13:41.664751 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:41.664755 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:41.664818 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:41.664928 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:41.666507 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.667078 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.668639 (+ 1561us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.668865 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.669003 (+ 138us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.671707 (+ 2704us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:41.671712 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:41.671715 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:41.671748 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:41.673390 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:41.673972 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:41.673977 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:41.673979 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:41.674075 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:41.674470 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:41.674473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:41.674475 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:41.674673 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:41.674781 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:41.675340 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 22:13:41.675484 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:13:41.676611 17404 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.662376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:35081 (local address 127.28.47.130:42179)
0811 22:13:41.662853 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:41.662936 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:41.664519 (+ 1583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:41.665123 (+ 604us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:41.665130 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:41.665564 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:41.666300 (+ 736us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:41.666324 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.667230 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.667236 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:41.668492 (+ 1256us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:41.668503 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.669051 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.669055 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:41.669286 (+ 231us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.669922 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:41.669940 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:41.671527 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:41.673559 (+ 2032us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:41.673566 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:41.673584 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:41.673857 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:41.674179 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:41.674182 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:41.674184 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:41.674363 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:41.674793 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:41.674801 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:41.675092 (+ 291us) client_negotiation.cc:769] Sending connection context
0811 22:13:41.675294 (+ 202us) client_negotiation.cc:240] Negotiation successful
0811 22:13:41.675653 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":117,"threads_started":1}
I20260811 22:13:41.678105 17402 heartbeater.cc:347] Connected to a master server at 127.28.47.190:35081
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
I20260811 22:13:41.678453 17402 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:41.679112 17402 heartbeater.cc:511] Master 127.28.47.190:35081 requested a full tablet report, sending...
I20260811 22:13:41.680409 17028 ts_manager.cc:194] Registered new tserver with Master: 5a9108e49fe04774a83dacdc27322d16 (127.28.47.130:38647)
I20260811 22:13:41.681069 17028 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:42179
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:13:41.696513 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35081
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:42917
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:41.815860 17407 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:41.816265 17407 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:41.816360 17407 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:41.820749 17407 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:41.820830 17407 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:41.820894 17407 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:13:41.826701 17407 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:42917
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:35081
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17407
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:41.827883 17407 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:41.828953 17407 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:41.839556 17412 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:41.839556 17413 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:41.839649 17415 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:41.839941 17407 server_base.cc:1034] running on GCE node
I20260811 22:13:41.840377 17407 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:41.841109 17407 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:41.842346 17407 hybrid_clock.cc:648] HybridClock initialized: now 1786486421842320 us; error 38 us; skew 500 ppm
Aug 11 22:13:41 dist-test-slave-vxj2 krb5kdc[16976](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486421, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:41.845639 17407 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:13:41.846647 17407 webserver.cc:460] Webserver started at http://127.28.47.131:35455/ using document root <none> and password file <none>
I20260811 22:13:41.847285 17407 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:41.847355 17407 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:41.847590 17407 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:41.849900 17407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "00e5cc24c0884e43912de14da725d9ed"
format_stamp: "Formatted at 2026-08-11 22:13:41 on dist-test-slave-vxj2"
server_key: "7679c7a9c809fce7e1436362e05fe6b0"
server_key_iv: "8018b5b1ba948d3178541e985498a962"
server_key_version: "encryptionkey@0"
I20260811 22:13:41.850528 17407 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "00e5cc24c0884e43912de14da725d9ed"
format_stamp: "Formatted at 2026-08-11 22:13:41 on dist-test-slave-vxj2"
server_key: "7679c7a9c809fce7e1436362e05fe6b0"
server_key_iv: "8018b5b1ba948d3178541e985498a962"
server_key_version: "encryptionkey@0"
I20260811 22:13:41.854861 17407 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:13:41.857867 17424 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:41.859289 17407 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:41.859433 17407 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "00e5cc24c0884e43912de14da725d9ed"
format_stamp: "Formatted at 2026-08-11 22:13:41 on dist-test-slave-vxj2"
server_key: "7679c7a9c809fce7e1436362e05fe6b0"
server_key_iv: "8018b5b1ba948d3178541e985498a962"
server_key_version: "encryptionkey@0"
I20260811 22:13:41.859557 17407 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:41.878566 17407 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:41.882192 17407 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:41.882423 17407 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:41.883103 17407 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:41.884065 17407 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:41.884115 17407 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:41.884182 17407 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:41.884220 17407 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:41.894374 17407 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:33847
I20260811 22:13:41.894456 17557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:33847 every 8 connection(s)
I20260811 22:13:41.895367 17407 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:13:41.904582 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 17407
I20260811 22:13:41.904747 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:13:41.905129 28862 external_mini_cluster.cc:1428] Setting key 5c53ed83e223d6cdcb694948ca75cc9a
Aug 11 22:13:41 dist-test-slave-vxj2 krb5kdc[16976](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486421, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:13:41.910300 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.897526 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:52309 (local address 127.28.47.190:35081)
0811 22:13:41.897680 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:41.897684 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:41.898535 (+ 851us) server_negotiation.cc:407] Connection header received
0811 22:13:41.899911 (+ 1376us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:41.899915 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:41.899981 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:41.900067 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:41.901315 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.901863 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.903104 (+ 1241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.903335 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.903455 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.906767 (+ 3312us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:41.906770 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:41.906773 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:41.906801 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:41.908278 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:41.908777 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:41.908779 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:41.908781 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:41.908859 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:41.909263 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:41.909269 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:41.909270 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:41.909468 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:41.909548 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:41.910070 (+ 522us) server_negotiation.cc:299] Negotiation successful
0811 22:13:41.910175 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:13:41.911337 17560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.897883 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:35081 (local address 127.28.47.131:52309)
0811 22:13:41.898362 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:41.898409 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:41.899679 (+ 1270us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:41.900245 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:41.900251 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:41.900619 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:41.901126 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:41.901140 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.902008 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.902013 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:41.902959 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:41.902970 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.903505 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.903510 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:41.903694 (+ 184us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.904441 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:41.904465 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:41.906575 (+ 2110us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:41.908388 (+ 1813us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:41.908394 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:41.908408 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:41.908684 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:41.908961 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:41.908964 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:41.908965 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:41.909106 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:41.909565 (+ 459us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:41.909570 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:41.909837 (+ 267us) client_negotiation.cc:769] Sending connection context
0811 22:13:41.910006 (+ 169us) client_negotiation.cc:240] Negotiation successful
0811 22:13:41.910292 (+ 286us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":125,"threads_started":1}
I20260811 22:13:41.912604 17558 heartbeater.cc:347] Connected to a master server at 127.28.47.190:35081
I20260811 22:13:41.912910 17558 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:41.913615 17558 heartbeater.cc:511] Master 127.28.47.190:35081 requested a full tablet report, sending...
I20260811 22:13:41.914764 17028 ts_manager.cc:194] Registered new tserver with Master: 00e5cc24c0884e43912de14da725d9ed (127.28.47.131:33847)
I20260811 22:13:41.915395 17028 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:52309
I20260811 22:13:41.919612 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:13:41.929824 17076 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.921763 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34134 (local address 127.28.47.190:35081)
0811 22:13:41.921889 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:41.921893 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:41.922051 (+ 158us) server_negotiation.cc:407] Connection header received
0811 22:13:41.922158 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:41.922160 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:41.922202 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:41.922272 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:41.923086 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.923606 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.924457 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.924658 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.924732 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.926245 (+ 1513us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:41.926247 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:41.926250 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:41.926275 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:41.927730 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:41.928137 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:41.928141 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:41.928142 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:41.928238 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:41.928494 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:41.928496 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:41.928498 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:41.928944 (+ 446us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:41.929105 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:41.929460 (+ 355us) server_negotiation.cc:299] Negotiation successful
0811 22:13:41.929599 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260811 22:13:41.932770 17028 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34134:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:13:41.936004 17028 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:13:41.951828 17571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.945266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:33847 (local address 127.0.0.1:53822)
0811 22:13:41.946051 (+ 785us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:41.946111 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:41.946252 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:41.947121 (+ 869us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:41.947128 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:41.947151 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:41.947332 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:41.947339 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.948893 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.948896 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:41.950033 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:41.950043 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.951235 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.951239 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:41.951354 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.951397 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 22:13:41.951525 (+ 128us) client_negotiation.cc:240] Negotiation successful
0811 22:13:41.951648 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":592,"mutex_wait_us":15,"thread_start_us":81,"threads_started":1}
I20260811 22:13:41.951853 17572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.945480 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53822 (local address 127.28.47.131:33847)
0811 22:13:41.946631 (+ 1151us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:41.946645 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:41.946672 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:13:41.946753 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:41.946758 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:41.946952 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:41.947132 (+ 180us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:41.947498 (+ 366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.948736 (+ 1238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.950165 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.951098 (+ 933us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.951209 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.951382 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:41.951579 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 22:13:41.951699 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1029,"thread_start_us":185,"threads_started":1}
I20260811 22:13:41.952385 17575 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.946026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:38647 (local address 127.0.0.1:53336)
0811 22:13:41.947648 (+ 1622us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:41.947665 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:41.947765 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:41.948206 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:41.948209 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:41.948227 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:41.948310 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:41.948315 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.949617 (+ 1302us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.949623 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:41.950642 (+ 1019us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:41.950654 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.951950 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.951954 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:41.952043 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.952066 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:13:41.952144 (+ 78us) client_negotiation.cc:240] Negotiation successful
0811 22:13:41.952228 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1544,"mutex_wait_us":72,"thread_start_us":115,"threads_started":1}
I20260811 22:13:41.952836 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.946316 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53336 (local address 127.28.47.130:38647)
0811 22:13:41.946895 (+ 579us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:41.946905 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:41.947779 (+ 874us) server_negotiation.cc:407] Connection header received
0811 22:13:41.947870 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:41.947883 (+ 13us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:41.948056 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:41.948198 (+ 142us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:41.948431 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.949495 (+ 1064us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.950791 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.951640 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.951742 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.951877 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:41.952587 (+ 710us) server_negotiation.cc:299] Negotiation successful
0811 22:13:41.952700 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":492,"thread_start_us":182,"threads_started":1}
I20260811 22:13:41.953220 17579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.946517 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39200 (local address 127.28.47.129:33815)
0811 22:13:41.947078 (+ 561us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:41.947082 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:41.947101 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:13:41.947171 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:41.947175 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:41.947318 (+ 143us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:41.947437 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:41.947703 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.948622 (+ 919us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.950918 (+ 2296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.951724 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.951827 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.952014 (+ 187us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:41.952792 (+ 778us) server_negotiation.cc:299] Negotiation successful
0811 22:13:41.952987 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":459,"thread_start_us":69,"threads_started":1}
I20260811 22:13:41.952688 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:41.946032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:33815 (local address 127.0.0.1:39200)
0811 22:13:41.946529 (+ 497us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:41.946545 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:41.946630 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:41.947445 (+ 815us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:41.947450 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:41.947472 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:41.947571 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:41.947577 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.948751 (+ 1174us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.948755 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:41.950790 (+ 2035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:41.950797 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:41.952272 (+ 1475us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:41.952281 (+ 9us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:41.952395 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:41.952419 (+ 24us) client_negotiation.cc:769] Sending connection context
0811 22:13:41.952490 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:13:41.952564 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":430,"mutex_wait_us":65,"thread_start_us":102,"threads_started":1}
I20260811 22:13:41.955789 17323 tablet_service.cc:1467] Processing CreateTablet for tablet 1084c07e168748d6960ea1223d30b3a9 (DEFAULT_TABLE table=test-table [id=255b8b747c684d888a8c57d7efb7b6cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:41.955790 17483 tablet_service.cc:1467] Processing CreateTablet for tablet 1084c07e168748d6960ea1223d30b3a9 (DEFAULT_TABLE table=test-table [id=255b8b747c684d888a8c57d7efb7b6cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:41.957068 17483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1084c07e168748d6960ea1223d30b3a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:41.957068 17323 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1084c07e168748d6960ea1223d30b3a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:41.959311 17173 tablet_service.cc:1467] Processing CreateTablet for tablet 1084c07e168748d6960ea1223d30b3a9 (DEFAULT_TABLE table=test-table [id=255b8b747c684d888a8c57d7efb7b6cd]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:41.960255 17173 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1084c07e168748d6960ea1223d30b3a9. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:41.965189 17582 tablet_bootstrap.cc:492] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed: Bootstrap starting.
I20260811 22:13:41.965679 17580 tablet_bootstrap.cc:492] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16: Bootstrap starting.
I20260811 22:13:41.967921 17583 tablet_bootstrap.cc:492] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc: Bootstrap starting.
I20260811 22:13:41.968271 17580 tablet_bootstrap.cc:654] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:41.968511 17582 tablet_bootstrap.cc:654] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:41.969520 17580 log.cc:824] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:41.969827 17582 log.cc:824] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:41.970340 17583 tablet_bootstrap.cc:654] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:41.971158 17583 log.cc:824] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:41.971589 17580 tablet_bootstrap.cc:492] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16: No bootstrap required, opened a new log
I20260811 22:13:41.971836 17580 ts_tablet_manager.cc:1397] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:13:41.972977 17583 tablet_bootstrap.cc:492] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc: No bootstrap required, opened a new log
I20260811 22:13:41.973213 17583 ts_tablet_manager.cc:1397] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.004s
I20260811 22:13:41.974942 17582 tablet_bootstrap.cc:492] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed: No bootstrap required, opened a new log
I20260811 22:13:41.975185 17582 ts_tablet_manager.cc:1397] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed: Time spent bootstrapping tablet: real 0.010s user 0.002s sys 0.002s
I20260811 22:13:41.975708 17580 raft_consensus.cc:348] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00e5cc24c0884e43912de14da725d9ed" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33847 } } peers { permanent_uuid: "5a9108e49fe04774a83dacdc27322d16" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38647 } } peers { permanent_uuid: "e2138d00b0e74a29ace51b72149bd3dc" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33815 } }
I20260811 22:13:41.975968 17580 raft_consensus.cc:374] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:41.976046 17580 raft_consensus.cc:729] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5a9108e49fe04774a83dacdc27322d16, State: Initialized, Role: FOLLOWER
I20260811 22:13:41.976536 17580 consensus_queue.cc:260] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00e5cc24c0884e43912de14da725d9ed" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33847 } } peers { permanent_uuid: "5a9108e49fe04774a83dacdc27322d16" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38647 } } peers { permanent_uuid: "e2138d00b0e74a29ace51b72149bd3dc" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33815 } }
I20260811 22:13:41.977295 17402 heartbeater.cc:503] Master 127.28.47.190:35081 was elected leader, sending a full tablet report...
I20260811 22:13:41.977882 17583 raft_consensus.cc:348] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00e5cc24c0884e43912de14da725d9ed" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33847 } } peers { permanent_uuid: "5a9108e49fe04774a83dacdc27322d16" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38647 } } peers { permanent_uuid: "e2138d00b0e74a29ace51b72149bd3dc" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33815 } }
I20260811 22:13:41.978245 17583 raft_consensus.cc:374] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:41.978344 17583 raft_consensus.cc:729] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2138d00b0e74a29ace51b72149bd3dc, State: Initialized, Role: FOLLOWER
I20260811 22:13:41.978729 17580 ts_tablet_manager.cc:1428] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:13:41.978919 17583 consensus_queue.cc:260] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00e5cc24c0884e43912de14da725d9ed" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33847 } } peers { permanent_uuid: "5a9108e49fe04774a83dacdc27322d16" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38647 } } peers { permanent_uuid: "e2138d00b0e74a29ace51b72149bd3dc" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33815 } }
I20260811 22:13:41.979184 17582 raft_consensus.cc:348] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00e5cc24c0884e43912de14da725d9ed" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33847 } } peers { permanent_uuid: "5a9108e49fe04774a83dacdc27322d16" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38647 } } peers { permanent_uuid: "e2138d00b0e74a29ace51b72149bd3dc" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33815 } }
I20260811 22:13:41.979511 17582 raft_consensus.cc:374] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:41.979605 17582 raft_consensus.cc:729] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00e5cc24c0884e43912de14da725d9ed, State: Initialized, Role: FOLLOWER
I20260811 22:13:41.979661 17249 heartbeater.cc:503] Master 127.28.47.190:35081 was elected leader, sending a full tablet report...
I20260811 22:13:41.980020 17583 ts_tablet_manager.cc:1428] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:13:41.980131 17582 consensus_queue.cc:260] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00e5cc24c0884e43912de14da725d9ed" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33847 } } peers { permanent_uuid: "5a9108e49fe04774a83dacdc27322d16" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38647 } } peers { permanent_uuid: "e2138d00b0e74a29ace51b72149bd3dc" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33815 } }
I20260811 22:13:41.980796 17558 heartbeater.cc:503] Master 127.28.47.190:35081 was elected leader, sending a full tablet report...
I20260811 22:13:41.981133 17582 ts_tablet_manager.cc:1428] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
W20260811 22:13:42.146513 17559 tablet.cc:2369] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:13:42.154668 17250 tablet.cc:2369] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260811 22:13:42.161182 17403 tablet.cc:2369] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:13:42.253961 17587 raft_consensus.cc:482] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:13:42.254208 17587 raft_consensus.cc:504] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00e5cc24c0884e43912de14da725d9ed" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33847 } } peers { permanent_uuid: "5a9108e49fe04774a83dacdc27322d16" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38647 } } peers { permanent_uuid: "e2138d00b0e74a29ace51b72149bd3dc" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33815 } }
I20260811 22:13:42.255448 17587 leader_election.cc:288] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 00e5cc24c0884e43912de14da725d9ed (127.28.47.131:33847), e2138d00b0e74a29ace51b72149bd3dc (127.28.47.129:33815)
I20260811 22:13:42.261559 17572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:42.256070 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:46281 (local address 127.28.47.131:33847)
0811 22:13:42.256248 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:42.256254 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:42.256300 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:13:42.256563 (+ 263us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:42.256568 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:42.256635 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:42.256717 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:42.257263 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.258075 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.260098 (+ 2023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.260769 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.260909 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:42.260981 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:42.261324 (+ 343us) server_negotiation.cc:299] Negotiation successful
0811 22:13:42.261408 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260811 22:13:42.261559 17579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:42.256433 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:33463 (local address 127.28.47.129:33815)
0811 22:13:42.256584 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:42.256588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:42.256701 (+ 113us) server_negotiation.cc:407] Connection header received
0811 22:13:42.256798 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:42.256801 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:42.256861 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:42.256996 (+ 135us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:42.257339 (+ 343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.258094 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.260098 (+ 2004us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.260772 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.260909 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:42.260973 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:42.261324 (+ 351us) server_negotiation.cc:299] Negotiation successful
0811 22:13:42.261409 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260811 22:13:42.261642 17590 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:42.255847 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:33847 (local address 127.28.47.130:46281)
0811 22:13:42.256177 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:42.256217 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:42.256358 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:42.256912 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:42.256915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:42.256936 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:42.257084 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:42.257092 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.258228 (+ 1136us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.258232 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:42.259922 (+ 1690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:42.259937 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.260930 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.260934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:42.261122 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:42.261213 (+ 91us) client_negotiation.cc:769] Sending connection context
0811 22:13:42.261308 (+ 95us) client_negotiation.cc:240] Negotiation successful
0811 22:13:42.261452 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":225,"mutex_wait_us":227,"thread_start_us":91,"threads_started":1}
I20260811 22:13:42.261644 17591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:42.256269 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:33815 (local address 127.28.47.130:33463)
0811 22:13:42.256594 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:42.256606 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:42.256711 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:42.257021 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:42.257024 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:42.257049 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:42.257209 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:42.257216 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.258231 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.258234 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:42.259926 (+ 1692us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:42.259937 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.260930 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.260934 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:42.261120 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:42.261213 (+ 93us) client_negotiation.cc:769] Sending connection context
0811 22:13:42.261302 (+ 89us) client_negotiation.cc:240] Negotiation successful
0811 22:13:42.261452 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":252,"mutex_wait_us":296,"thread_start_us":118,"threads_started":1}
I20260811 22:13:42.262542 17196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1084c07e168748d6960ea1223d30b3a9" candidate_uuid: "5a9108e49fe04774a83dacdc27322d16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2138d00b0e74a29ace51b72149bd3dc" is_pre_election: true
I20260811 22:13:42.262603 17505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1084c07e168748d6960ea1223d30b3a9" candidate_uuid: "5a9108e49fe04774a83dacdc27322d16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00e5cc24c0884e43912de14da725d9ed" is_pre_election: true
I20260811 22:13:42.262923 17196 raft_consensus.cc:2454] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a9108e49fe04774a83dacdc27322d16 in term 0.
I20260811 22:13:42.262930 17505 raft_consensus.cc:2454] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5a9108e49fe04774a83dacdc27322d16 in term 0.
I20260811 22:13:42.263510 17275 leader_election.cc:302] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5a9108e49fe04774a83dacdc27322d16, e2138d00b0e74a29ace51b72149bd3dc; no voters:
I20260811 22:13:42.263833 17587 raft_consensus.cc:2781] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:13:42.263923 17587 raft_consensus.cc:482] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:13:42.263955 17587 raft_consensus.cc:3037] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:42.265225 17587 raft_consensus.cc:504] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00e5cc24c0884e43912de14da725d9ed" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33847 } } peers { permanent_uuid: "5a9108e49fe04774a83dacdc27322d16" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38647 } } peers { permanent_uuid: "e2138d00b0e74a29ace51b72149bd3dc" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33815 } }
I20260811 22:13:42.265627 17587 leader_election.cc:288] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [CANDIDATE]: Term 1 election: Requested vote from peers 00e5cc24c0884e43912de14da725d9ed (127.28.47.131:33847), e2138d00b0e74a29ace51b72149bd3dc (127.28.47.129:33815)
I20260811 22:13:42.266160 17505 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1084c07e168748d6960ea1223d30b3a9" candidate_uuid: "5a9108e49fe04774a83dacdc27322d16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00e5cc24c0884e43912de14da725d9ed"
I20260811 22:13:42.266302 17505 raft_consensus.cc:3037] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:42.266373 17196 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1084c07e168748d6960ea1223d30b3a9" candidate_uuid: "5a9108e49fe04774a83dacdc27322d16" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2138d00b0e74a29ace51b72149bd3dc"
I20260811 22:13:42.266561 17196 raft_consensus.cc:3037] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:42.267573 17505 raft_consensus.cc:2454] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a9108e49fe04774a83dacdc27322d16 in term 1.
I20260811 22:13:42.267727 17196 raft_consensus.cc:2454] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5a9108e49fe04774a83dacdc27322d16 in term 1.
I20260811 22:13:42.267987 17275 leader_election.cc:302] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00e5cc24c0884e43912de14da725d9ed, 5a9108e49fe04774a83dacdc27322d16; no voters:
I20260811 22:13:42.268169 17587 raft_consensus.cc:2781] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:42.268435 17587 raft_consensus.cc:686] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [term 1 LEADER]: Becoming Leader. State: Replica: 5a9108e49fe04774a83dacdc27322d16, State: Running, Role: LEADER
I20260811 22:13:42.268850 17587 consensus_queue.cc:237] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00e5cc24c0884e43912de14da725d9ed" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33847 } } peers { permanent_uuid: "5a9108e49fe04774a83dacdc27322d16" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38647 } } peers { permanent_uuid: "e2138d00b0e74a29ace51b72149bd3dc" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33815 } }
I20260811 22:13:42.272521 17028 catalog_manager.cc:5495] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5a9108e49fe04774a83dacdc27322d16 (127.28.47.130). New cstate: current_term: 1 leader_uuid: "5a9108e49fe04774a83dacdc27322d16" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "00e5cc24c0884e43912de14da725d9ed" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33847 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5a9108e49fe04774a83dacdc27322d16" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 38647 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e2138d00b0e74a29ace51b72149bd3dc" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33815 } health_report { overall_health: UNKNOWN } } }
I20260811 22:13:42.317903 17578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:42.313994 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53338 (local address 127.28.47.130:38647)
0811 22:13:42.314140 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:42.314144 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:42.314158 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:13:42.314243 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:42.314245 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:42.314307 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:42.314408 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:13:42.314649 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.315174 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.316155 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.316432 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.316529 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:42.316838 (+ 309us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:13:42.317405 (+ 567us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:13:42.317536 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:42.317656 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 22:13:42.317715 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:13:42.324421 17196 raft_consensus.cc:1261] T 1084c07e168748d6960ea1223d30b3a9 P e2138d00b0e74a29ace51b72149bd3dc [term 1 FOLLOWER]: Refusing update from remote peer 5a9108e49fe04774a83dacdc27322d16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:42.324421 17505 raft_consensus.cc:1261] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed [term 1 FOLLOWER]: Refusing update from remote peer 5a9108e49fe04774a83dacdc27322d16: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:42.325241 17587 consensus_queue.cc:1035] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "00e5cc24c0884e43912de14da725d9ed" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33847 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:42.325470 17592 consensus_queue.cc:1035] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2138d00b0e74a29ace51b72149bd3dc" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 33815 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:42.343688 17028 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:42.343870 17028 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34134:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:42.346798 17028 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 3de90885e8354b93830953f58eeb06c4: Sending DeleteTablet for 3 replicas of tablet 1084c07e168748d6960ea1223d30b3a9
I20260811 22:13:42.347853 17483 tablet_service.cc:1514] Processing DeleteTablet for tablet 1084c07e168748d6960ea1223d30b3a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:42 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:53822
I20260811 22:13:42.348289 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 17101
I20260811 22:13:42.348507 17605 tablet_replica.cc:330] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed: stopping tablet replica
I20260811 22:13:42.348642 17323 tablet_service.cc:1514] Processing DeleteTablet for tablet 1084c07e168748d6960ea1223d30b3a9 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:42 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:53336
I20260811 22:13:42.348894 17605 raft_consensus.cc:2229] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:42.349375 17605 raft_consensus.cc:2258] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:42.350359 17606 tablet_replica.cc:330] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16: stopping tablet replica
I20260811 22:13:42.350817 17606 raft_consensus.cc:2229] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:13:42.351492 17606 raft_consensus.cc:2258] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:42.354092 17605 ts_tablet_manager.cc:1905] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:42.354132 17606 ts_tablet_manager.cc:1905] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:42.357810 17605 ts_tablet_manager.cc:1918] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:42.357985 17605 log.cc:1196] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/1084c07e168748d6960ea1223d30b3a9
I20260811 22:13:42.358392 17605 ts_tablet_manager.cc:1939] T 1084c07e168748d6960ea1223d30b3a9 P 00e5cc24c0884e43912de14da725d9ed: Deleting consensus metadata
I20260811 22:13:42.358847 17606 ts_tablet_manager.cc:1918] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:42.359052 17606 log.cc:1196] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/1084c07e168748d6960ea1223d30b3a9
I20260811 22:13:42.359437 17606 ts_tablet_manager.cc:1939] T 1084c07e168748d6960ea1223d30b3a9 P 5a9108e49fe04774a83dacdc27322d16: Deleting consensus metadata
I20260811 22:13:42.361799 17012 catalog_manager.cc:4841] TS 5a9108e49fe04774a83dacdc27322d16 (127.28.47.130:38647): tablet 1084c07e168748d6960ea1223d30b3a9 (table test-table [id=255b8b747c684d888a8c57d7efb7b6cd]) successfully deleted
I20260811 22:13:42.369148 17014 catalog_manager.cc:4841] TS 00e5cc24c0884e43912de14da725d9ed (127.28.47.131:33847): tablet 1084c07e168748d6960ea1223d30b3a9 (table test-table [id=255b8b747c684d888a8c57d7efb7b6cd]) successfully deleted
W20260811 22:13:42.372288 17014 connection.cc:535] client connection to 127.28.47.129:33815 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260811 22:13:42.372506 17014 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260811 22:13:42.373462 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 17256
I20260811 22:13:42.373832 17574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:42.373505 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:33815 (local address 127.0.0.1:39204)
0811 22:13:42.373640 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:42.373722 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:33815: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260811 22:13:42.381651 17014 catalog_manager.cc:4584] TS e2138d00b0e74a29ace51b72149bd3dc (127.28.47.129:33815): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1084c07e168748d6960ea1223d30b3a9: Network error: Client connection negotiation failed: client connection to 127.28.47.129:33815: connect: Connection refused (error 111)
I20260811 22:13:42.387727 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 17407
I20260811 22:13:42.396296 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 16993
2026-08-11T22:13:42Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2806 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-11T22:13:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:42Z Disabled control of system clock
I20260811 22:13:42.433848 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:46771
--webserver_interface=127.28.47.190
--webserver_port=0
--builtin_ntp_servers=127.28.47.148:33083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:46771
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:13:42.554337 17611 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:42.554651 17611 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:42.554689 17611 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:42.558895 17611 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:42.558977 17611 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:42.558993 17611 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:42.559010 17611 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:42.559031 17611 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:42.564131 17611 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:46771
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:46771
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17611
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:42.565376 17611 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:42.566869 17611 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:42.576052 17619 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:42.576009 17617 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:42.576009 17616 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:42.576393 17611 server_base.cc:1034] running on GCE node
I20260811 22:13:42.577091 17611 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:42.578413 17611 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:42.579643 17611 hybrid_clock.cc:648] HybridClock initialized: now 1786486422579611 us; error 54 us; skew 500 ppm
I20260811 22:13:42.587436 17611 webserver.cc:460] Webserver started at http://127.28.47.190:41023/ using document root <none> and password file <none>
I20260811 22:13:42.588219 17611 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:42.588335 17611 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:42.588600 17611 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:42.591037 17611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "9527da75f7d24805843ffc3434e54a33"
format_stamp: "Formatted at 2026-08-11 22:13:42 on dist-test-slave-vxj2"
server_key: "2046ac5ceff7039d500d135aa1ff5300"
server_key_iv: "e4cb16af52fa0cf8ad74dcf320606f32"
server_key_version: "encryptionkey@0"
I20260811 22:13:42.591622 17611 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "9527da75f7d24805843ffc3434e54a33"
format_stamp: "Formatted at 2026-08-11 22:13:42 on dist-test-slave-vxj2"
server_key: "2046ac5ceff7039d500d135aa1ff5300"
server_key_iv: "e4cb16af52fa0cf8ad74dcf320606f32"
server_key_version: "encryptionkey@0"
I20260811 22:13:42.595685 17611 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:13:42.598474 17626 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:42.599822 17611 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:13:42.599989 17611 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "9527da75f7d24805843ffc3434e54a33"
format_stamp: "Formatted at 2026-08-11 22:13:42 on dist-test-slave-vxj2"
server_key: "2046ac5ceff7039d500d135aa1ff5300"
server_key_iv: "e4cb16af52fa0cf8ad74dcf320606f32"
server_key_version: "encryptionkey@0"
I20260811 22:13:42.600116 17611 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:42.624596 17611 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:42.625650 17611 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:42.625903 17611 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:42.634018 17611 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:46771
I20260811 22:13:42.634024 17688 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:46771 every 8 connection(s)
I20260811 22:13:42.635046 17611 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:42.638394 17689 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:42.641618 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 17611
I20260811 22:13:42.641784 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:13:42.642063 28862 external_mini_cluster.cc:1428] Setting key 0a6c8676c5dd29b77a2739708bd5792a
I20260811 22:13:42.644729 17689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33: Bootstrap starting.
I20260811 22:13:42.647172 17689 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:42.647953 17689 log.cc:824] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:42.649981 17689 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33: No bootstrap required, opened a new log
I20260811 22:13:42.651654 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:42.643570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60308 (local address 127.28.47.190:46771)
0811 22:13:42.644275 (+ 705us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:42.644291 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:42.644329 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:13:42.645046 (+ 717us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:42.645066 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:42.645546 (+ 480us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:42.645883 (+ 337us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:42.646354 (+ 471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.647385 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.648334 (+ 949us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.648816 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.649000 (+ 184us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:42.649902 (+ 902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:42.649907 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:42.649926 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:42.650365 (+ 439us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:42.650425 (+ 60us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:42.650447 (+ 22us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:42.650561 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:42.650677 (+ 116us) server_negotiation.cc:299] Negotiation successful
0811 22:13:42.650950 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":333,"thread_start_us":156,"threads_started":1}
I20260811 22:13:42.653733 17689 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9527da75f7d24805843ffc3434e54a33" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46771 } }
I20260811 22:13:42.653992 17689 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:42.654073 17689 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9527da75f7d24805843ffc3434e54a33, State: Initialized, Role: FOLLOWER
I20260811 22:13:42.654594 17689 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9527da75f7d24805843ffc3434e54a33" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46771 } }
I20260811 22:13:42.654775 17689 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:42.654850 17689 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:42.654946 17689 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:42.655953 17689 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9527da75f7d24805843ffc3434e54a33" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46771 } }
I20260811 22:13:42.656306 17689 leader_election.cc:302] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9527da75f7d24805843ffc3434e54a33; no voters:
I20260811 22:13:42.656646 17689 leader_election.cc:288] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:42.656774 17695 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:42.657029 17695 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [term 1 LEADER]: Becoming Leader. State: Replica: 9527da75f7d24805843ffc3434e54a33, State: Running, Role: LEADER
I20260811 22:13:42.657434 17695 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9527da75f7d24805843ffc3434e54a33" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46771 } }
I20260811 22:13:42.658046 17689 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:42.659228 17697 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9527da75f7d24805843ffc3434e54a33. Latest consensus state: current_term: 1 leader_uuid: "9527da75f7d24805843ffc3434e54a33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9527da75f7d24805843ffc3434e54a33" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46771 } } }
I20260811 22:13:42.659387 17697 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:42.659662 17696 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9527da75f7d24805843ffc3434e54a33" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9527da75f7d24805843ffc3434e54a33" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 46771 } } }
I20260811 22:13:42.659766 17705 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:42.659785 17696 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:42.663235 17705 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:42.669538 17705 catalog_manager.cc:1347] Generated new cluster ID: 199d21ce33e84e94944665f8b3e93add
I20260811 22:13:42.669633 17705 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:42.693317 17705 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:42.694289 17705 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:42.708892 17705 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33: Generated new TSK 0
I20260811 22:13:42.709759 17705 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:13:42.717698 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46771
--builtin_ntp_servers=127.28.47.148:33083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:13:42.854118 17718 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:42.854429 17718 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:42.854472 17718 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:42.858639 17718 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:42.858708 17718 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:42.858769 17718 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:13:42.863879 17718 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17718
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:42.865037 17718 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:42.866616 17718 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:42.877130 17726 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:42.877138 17723 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:42.877143 17724 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:42.877342 17718 server_base.cc:1034] running on GCE node
I20260811 22:13:42.878607 17718 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:42.879338 17718 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:42.880568 17718 hybrid_clock.cc:648] HybridClock initialized: now 1786486422880544 us; error 44 us; skew 500 ppm
I20260811 22:13:42.882865 17718 webserver.cc:460] Webserver started at http://127.28.47.129:46637/ using document root <none> and password file <none>
I20260811 22:13:42.883495 17718 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:42.883597 17718 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:42.883849 17718 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:42.886209 17718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "3521c69481e4451c9795cba07df893be"
format_stamp: "Formatted at 2026-08-11 22:13:42 on dist-test-slave-vxj2"
server_key: "886522362006639aaec81b30d3d8147e"
server_key_iv: "cdc10faca5feb56c7611ac30cae52cc2"
server_key_version: "encryptionkey@0"
I20260811 22:13:42.886947 17718 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "3521c69481e4451c9795cba07df893be"
format_stamp: "Formatted at 2026-08-11 22:13:42 on dist-test-slave-vxj2"
server_key: "886522362006639aaec81b30d3d8147e"
server_key_iv: "cdc10faca5feb56c7611ac30cae52cc2"
server_key_version: "encryptionkey@0"
I20260811 22:13:42.891135 17718 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:13:42.893929 17736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:42.895303 17718 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:42.895478 17718 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "3521c69481e4451c9795cba07df893be"
format_stamp: "Formatted at 2026-08-11 22:13:42 on dist-test-slave-vxj2"
server_key: "886522362006639aaec81b30d3d8147e"
server_key_iv: "cdc10faca5feb56c7611ac30cae52cc2"
server_key_version: "encryptionkey@0"
I20260811 22:13:42.895617 17718 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:42.915879 17718 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:42.916728 17718 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:42.916990 17718 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:42.917701 17718 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:42.918773 17718 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:42.918850 17718 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:42.918927 17718 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:42.918978 17718 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:42.929955 17718 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:38009
I20260811 22:13:42.929968 17868 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:38009 every 8 connection(s)
I20260811 22:13:42.931223 17718 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:13:42.936856 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 17718
I20260811 22:13:42.937043 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:13:42.937395 28862 external_mini_cluster.cc:1428] Setting key a24f081c0a2c49b084e2311af9f23e54
I20260811 22:13:42.939653 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46771
--builtin_ntp_servers=127.28.47.148:33083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:13:42.942441 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:42.933636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:49273 (local address 127.28.47.190:46771)
0811 22:13:42.933802 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:42.933806 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:42.935006 (+ 1200us) server_negotiation.cc:407] Connection header received
0811 22:13:42.936535 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:42.936538 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:42.936593 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:42.936678 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:42.938145 (+ 1467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.938773 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.940209 (+ 1436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.940461 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.940583 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:42.941925 (+ 1342us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:42.941929 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:42.941931 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:42.941945 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:42.941967 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:42.941977 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:42.942059 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:42.942260 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 22:13:42.942320 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:13:42.943454 17871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:42.934095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:46771 (local address 127.28.47.129:49273)
0811 22:13:42.934854 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:42.934899 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:42.936257 (+ 1358us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:42.936934 (+ 677us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:42.936946 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:42.937081 (+ 135us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:42.937917 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:42.937937 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.938990 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.938992 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:42.940053 (+ 1061us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:42.940064 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:42.940640 (+ 576us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:42.940645 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:42.940831 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:42.941570 (+ 739us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:42.941574 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:42.941637 (+ 63us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:42.941737 (+ 100us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:42.942080 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:42.942094 (+ 14us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:42.942102 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:13:42.942252 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 22:13:42.942613 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":136,"threads_started":1}
I20260811 22:13:42.944934 17869 heartbeater.cc:347] Connected to a master server at 127.28.47.190:46771
I20260811 22:13:42.945304 17869 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:42.945955 17869 heartbeater.cc:511] Master 127.28.47.190:46771 requested a full tablet report, sending...
I20260811 22:13:42.947790 17646 ts_manager.cc:194] Registered new tserver with Master: 3521c69481e4451c9795cba07df893be (127.28.47.129:38009)
I20260811 22:13:42.949100 17646 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.47.129:49273
W20260811 22:13:43.063733 17872 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:43.064085 17872 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:43.064159 17872 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:43.068567 17872 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:43.068673 17872 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:43.068835 17872 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:13:43.074023 17872 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17872
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:43.075258 17872 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:43.076838 17872 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:43.087610 17879 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:43.087627 17878 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:43.087702 17872 server_base.cc:1034] running on GCE node
W20260811 22:13:43.087639 17882 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:43.088375 17872 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:43.089205 17872 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:43.090499 17872 hybrid_clock.cc:648] HybridClock initialized: now 1786486423090450 us; error 49 us; skew 500 ppm
I20260811 22:13:43.092677 17872 webserver.cc:460] Webserver started at http://127.28.47.130:33937/ using document root <none> and password file <none>
I20260811 22:13:43.093430 17872 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:43.093533 17872 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:43.093792 17872 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:43.096096 17872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "1eee4251762342c494192fa26ea7a18f"
format_stamp: "Formatted at 2026-08-11 22:13:43 on dist-test-slave-vxj2"
server_key: "693b11b13242a0595d1011550fb6ec38"
server_key_iv: "b35e2c3d896cad687a5c2bb5af8f9f19"
server_key_version: "encryptionkey@0"
I20260811 22:13:43.096710 17872 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "1eee4251762342c494192fa26ea7a18f"
format_stamp: "Formatted at 2026-08-11 22:13:43 on dist-test-slave-vxj2"
server_key: "693b11b13242a0595d1011550fb6ec38"
server_key_iv: "b35e2c3d896cad687a5c2bb5af8f9f19"
server_key_version: "encryptionkey@0"
I20260811 22:13:43.101009 17872 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:13:43.103888 17888 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:43.105636 17872 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:43.105826 17872 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "1eee4251762342c494192fa26ea7a18f"
format_stamp: "Formatted at 2026-08-11 22:13:43 on dist-test-slave-vxj2"
server_key: "693b11b13242a0595d1011550fb6ec38"
server_key_iv: "b35e2c3d896cad687a5c2bb5af8f9f19"
server_key_version: "encryptionkey@0"
I20260811 22:13:43.105973 17872 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:43.121730 17872 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:43.122709 17872 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:43.122944 17872 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:43.123590 17872 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:43.124636 17872 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:43.124709 17872 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:43.124775 17872 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:43.124812 17872 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:43.134582 17872 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:37387
I20260811 22:13:43.134601 18021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:37387 every 8 connection(s)
I20260811 22:13:43.135586 17872 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:13:43.138255 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 17872
I20260811 22:13:43.138417 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:13:43.138795 28862 external_mini_cluster.cc:1428] Setting key 43113b9b18688a73773a3b7f259cc612
I20260811 22:13:43.141360 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46771
--builtin_ntp_servers=127.28.47.148:33083
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:13:43.146883 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.137623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:44057 (local address 127.28.47.190:46771)
0811 22:13:43.137812 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:43.137819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:43.139224 (+ 1405us) server_negotiation.cc:407] Connection header received
0811 22:13:43.141012 (+ 1788us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:43.141015 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:43.141066 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:43.141211 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:43.142544 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.143081 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.144375 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.144616 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.144734 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.146200 (+ 1466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:43.146208 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:43.146214 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:43.146236 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:43.146266 (+ 30us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:43.146281 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:43.146374 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:43.146561 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 22:13:43.146650 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260811 22:13:43.147826 18024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.138203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:46771 (local address 127.28.47.130:44057)
0811 22:13:43.139063 (+ 860us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:43.139109 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:43.140735 (+ 1626us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:43.141424 (+ 689us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:43.141434 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:43.141559 (+ 125us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:43.142330 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.142355 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.143254 (+ 899us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.143257 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.144231 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.144243 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.144798 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.144803 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.144979 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.145774 (+ 795us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:43.145775 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:43.145859 (+ 84us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:43.145974 (+ 115us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:43.146385 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:43.146394 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:43.146398 (+ 4us) client_negotiation.cc:769] Sending connection context
0811 22:13:43.146523 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:13:43.146843 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":184,"threads_started":1}
I20260811 22:13:43.149449 18022 heartbeater.cc:347] Connected to a master server at 127.28.47.190:46771
I20260811 22:13:43.149740 18022 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:43.150386 18022 heartbeater.cc:511] Master 127.28.47.190:46771 requested a full tablet report, sending...
I20260811 22:13:43.151654 17646 ts_manager.cc:194] Registered new tserver with Master: 1eee4251762342c494192fa26ea7a18f (127.28.47.130:37387)
I20260811 22:13:43.152281 17646 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.47.130:44057
W20260811 22:13:43.263984 18025 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:43.264343 18025 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:43.264423 18025 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:43.268726 18025 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:43.268836 18025 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:43.268941 18025 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:13:43.274061 18025 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:33083
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:46771
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18025
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:43.275398 18025 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:43.276957 18025 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:43.287097 18030 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:43.287177 18031 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:43.287177 18033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:43.287602 18025 server_base.cc:1034] running on GCE node
I20260811 22:13:43.288043 18025 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:43.288751 18025 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:43.289952 18025 hybrid_clock.cc:648] HybridClock initialized: now 1786486423289932 us; error 32 us; skew 500 ppm
I20260811 22:13:43.291873 18025 webserver.cc:460] Webserver started at http://127.28.47.131:46205/ using document root <none> and password file <none>
I20260811 22:13:43.292481 18025 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:43.292582 18025 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:43.292835 18025 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:43.295070 18025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "a72b6dfbdd4d4603869c41702ed0af75"
format_stamp: "Formatted at 2026-08-11 22:13:43 on dist-test-slave-vxj2"
server_key: "16731c8c047cd986d5316c611685c5be"
server_key_iv: "213d4114af8e283e0729745203634f55"
server_key_version: "encryptionkey@0"
I20260811 22:13:43.295672 18025 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "a72b6dfbdd4d4603869c41702ed0af75"
format_stamp: "Formatted at 2026-08-11 22:13:43 on dist-test-slave-vxj2"
server_key: "16731c8c047cd986d5316c611685c5be"
server_key_iv: "213d4114af8e283e0729745203634f55"
server_key_version: "encryptionkey@0"
I20260811 22:13:43.299779 18025 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:13:43.302623 18039 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:43.304126 18025 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:13:43.304320 18025 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "a72b6dfbdd4d4603869c41702ed0af75"
format_stamp: "Formatted at 2026-08-11 22:13:43 on dist-test-slave-vxj2"
server_key: "16731c8c047cd986d5316c611685c5be"
server_key_iv: "213d4114af8e283e0729745203634f55"
server_key_version: "encryptionkey@0"
I20260811 22:13:43.304461 18025 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:43.320437 18025 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:43.321292 18025 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:43.321527 18025 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:43.322278 18025 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:43.323422 18025 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:43.323505 18025 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:43.323591 18025 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:43.323639 18025 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:43.334098 18025 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:34891
I20260811 22:13:43.334123 18169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:34891 every 8 connection(s)
I20260811 22:13:43.335224 18025 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:13:43.339730 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 18025
I20260811 22:13:43.339826 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:13:43.340116 28862 external_mini_cluster.cc:1428] Setting key 3c5936a62e56f3acff1b464b3cafef94
I20260811 22:13:43.345506 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.337340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:48325 (local address 127.28.47.190:46771)
0811 22:13:43.337512 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:43.337518 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:43.338626 (+ 1108us) server_negotiation.cc:407] Connection header received
0811 22:13:43.340152 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:43.340157 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:43.340218 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:43.340328 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:43.341661 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.342289 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.343500 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.343733 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.343814 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.344983 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:43.344986 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:43.344988 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:43.345001 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:43.345020 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:43.345029 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:43.345087 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:43.345316 (+ 229us) server_negotiation.cc:299] Negotiation successful
0811 22:13:43.345376 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:13:43.346525 18173 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.337749 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:46771 (local address 127.28.47.131:48325)
0811 22:13:43.338462 (+ 713us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:43.338515 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:43.339926 (+ 1411us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:43.340559 (+ 633us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:43.340569 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:43.340693 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:43.341486 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.341526 (+ 40us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.342428 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.342431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.343384 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.343394 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.343895 (+ 501us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.343898 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.344065 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.344694 (+ 629us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:43.344696 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:43.344749 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:43.344821 (+ 72us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:43.345105 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:43.345113 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:43.345118 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:13:43.345310 (+ 192us) client_negotiation.cc:240] Negotiation successful
0811 22:13:43.345583 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":126,"threads_started":1}
I20260811 22:13:43.347831 18171 heartbeater.cc:347] Connected to a master server at 127.28.47.190:46771
I20260811 22:13:43.348109 18171 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:43.348707 18171 heartbeater.cc:511] Master 127.28.47.190:46771 requested a full tablet report, sending...
I20260811 22:13:43.350044 17646 ts_manager.cc:194] Registered new tserver with Master: a72b6dfbdd4d4603869c41702ed0af75 (127.28.47.131:34891)
I20260811 22:13:43.350759 17646 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.47.131:48325
I20260811 22:13:43.354192 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:13:43.360942 17692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.356393 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60312 (local address 127.28.47.190:46771)
0811 22:13:43.356575 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:43.356582 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:43.356768 (+ 186us) server_negotiation.cc:407] Connection header received
0811 22:13:43.356916 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:43.356919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:43.356978 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:43.357077 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:43.357456 (+ 379us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.358063 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.358975 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.359286 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.359412 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.360424 (+ 1012us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:43.360428 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:43.360431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:43.360446 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:43.360477 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:43.360488 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:43.360574 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:43.360689 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 22:13:43.360764 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:13:43.364172 17646 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:60312:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:13:43.367347 17646 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:13:43.383585 18189 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.376933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:38009 (local address 127.0.0.1:44422)
0811 22:13:43.377420 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:43.377433 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:43.377536 (+ 103us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:43.378197 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:43.378200 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:43.378220 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:43.378334 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.378344 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.380162 (+ 1818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.380165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.381429 (+ 1264us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.381440 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.382845 (+ 1405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.382849 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.383022 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.383109 (+ 87us) client_negotiation.cc:769] Sending connection context
0811 22:13:43.383230 (+ 121us) client_negotiation.cc:240] Negotiation successful
0811 22:13:43.383404 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":431,"mutex_wait_us":66,"thread_start_us":91,"threads_started":1}
I20260811 22:13:43.384120 18190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.377371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44422 (local address 127.28.47.129:38009)
0811 22:13:43.377759 (+ 388us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:43.377772 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:43.377797 (+ 25us) server_negotiation.cc:407] Connection header received
0811 22:13:43.377879 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:43.377884 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:43.378048 (+ 164us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:43.378218 (+ 170us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:43.378500 (+ 282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.379922 (+ 1422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.381577 (+ 1655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.382438 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.382547 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.382779 (+ 232us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:43.383807 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0811 22:13:43.383945 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":136,"threads_started":1}
I20260811 22:13:43.384596 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.375940 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:37387 (local address 127.0.0.1:35692)
0811 22:13:43.377010 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:43.377066 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:43.377301 (+ 235us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:43.378008 (+ 707us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:43.378013 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:43.378035 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:43.378180 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.378186 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.379486 (+ 1300us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.379494 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.381076 (+ 1582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.381093 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.382799 (+ 1706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.382806 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.383020 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.383109 (+ 89us) client_negotiation.cc:769] Sending connection context
0811 22:13:43.383234 (+ 125us) client_negotiation.cc:240] Negotiation successful
0811 22:13:43.384418 (+ 1184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":894,"mutex_wait_us":1090,"thread_start_us":92,"threads_started":1}
I20260811 22:13:43.385008 18188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.376358 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53110 (local address 127.28.47.131:34891)
0811 22:13:43.377077 (+ 719us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:43.377089 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:43.377850 (+ 761us) server_negotiation.cc:407] Connection header received
0811 22:13:43.377993 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:43.377999 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:43.378191 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:43.378342 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:43.378696 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.379827 (+ 1131us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.382138 (+ 2311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.383842 (+ 1704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.383954 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.384264 (+ 310us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:43.384695 (+ 431us) server_negotiation.cc:299] Negotiation successful
0811 22:13:43.384831 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":593,"thread_start_us":133,"threads_started":1}
I20260811 22:13:43.384930 18185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.376164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:34891 (local address 127.0.0.1:53110)
0811 22:13:43.377258 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:43.377694 (+ 436us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:43.377866 (+ 172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:43.378420 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:43.378426 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:43.378464 (+ 38us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:43.378559 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.378565 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.379985 (+ 1420us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.379991 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.381946 (+ 1955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.381956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.384389 (+ 2433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.384392 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.384510 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.384561 (+ 51us) client_negotiation.cc:769] Sending connection context
0811 22:13:43.384632 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:13:43.384793 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1020,"mutex_wait_us":266,"thread_start_us":116,"threads_started":1}
I20260811 22:13:43.386852 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.376135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35692 (local address 127.28.47.130:37387)
0811 22:13:43.376847 (+ 712us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:43.376862 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:43.377182 (+ 320us) server_negotiation.cc:407] Connection header received
0811 22:13:43.377519 (+ 337us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:43.377526 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:43.377754 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:43.377921 (+ 167us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:43.378345 (+ 424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.379333 (+ 988us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.381285 (+ 1952us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.382323 (+ 1038us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.382454 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.382705 (+ 251us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:43.383373 (+ 668us) server_negotiation.cc:299] Negotiation successful
0811 22:13:43.383510 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":161,"threads_started":1}
I20260811 22:13:43.389753 17790 tablet_service.cc:1467] Processing CreateTablet for tablet ac603fb5b5f54c789e775ed28c36e91f (DEFAULT_TABLE table=test-table [id=2fde2321a9574dea85d7ba5172e703cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:43.389735 18093 tablet_service.cc:1467] Processing CreateTablet for tablet ac603fb5b5f54c789e775ed28c36e91f (DEFAULT_TABLE table=test-table [id=2fde2321a9574dea85d7ba5172e703cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:43.390942 18093 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac603fb5b5f54c789e775ed28c36e91f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:43.390942 17790 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac603fb5b5f54c789e775ed28c36e91f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:43.396219 17939 tablet_service.cc:1467] Processing CreateTablet for tablet ac603fb5b5f54c789e775ed28c36e91f (DEFAULT_TABLE table=test-table [id=2fde2321a9574dea85d7ba5172e703cb]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:43.397451 17939 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet ac603fb5b5f54c789e775ed28c36e91f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:43.398002 18191 tablet_bootstrap.cc:492] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be: Bootstrap starting.
I20260811 22:13:43.398669 18192 tablet_bootstrap.cc:492] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75: Bootstrap starting.
I20260811 22:13:43.401058 18191 tablet_bootstrap.cc:654] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:43.401113 18192 tablet_bootstrap.cc:654] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:43.401993 18191 log.cc:824] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:43.402007 18192 log.cc:824] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:43.404121 18191 tablet_bootstrap.cc:492] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be: No bootstrap required, opened a new log
I20260811 22:13:43.404146 18192 tablet_bootstrap.cc:492] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75: No bootstrap required, opened a new log
I20260811 22:13:43.404352 18192 ts_tablet_manager.cc:1397] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:13:43.404369 18191 ts_tablet_manager.cc:1397] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 22:13:43.406169 18196 tablet_bootstrap.cc:492] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f: Bootstrap starting.
I20260811 22:13:43.408270 18191 raft_consensus.cc:348] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eee4251762342c494192fa26ea7a18f" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37387 } } peers { permanent_uuid: "a72b6dfbdd4d4603869c41702ed0af75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 34891 } } peers { permanent_uuid: "3521c69481e4451c9795cba07df893be" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 38009 } }
I20260811 22:13:43.408607 18191 raft_consensus.cc:374] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:43.408667 18191 raft_consensus.cc:729] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3521c69481e4451c9795cba07df893be, State: Initialized, Role: FOLLOWER
I20260811 22:13:43.408860 18196 tablet_bootstrap.cc:654] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:43.408871 18192 raft_consensus.cc:348] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eee4251762342c494192fa26ea7a18f" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37387 } } peers { permanent_uuid: "a72b6dfbdd4d4603869c41702ed0af75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 34891 } } peers { permanent_uuid: "3521c69481e4451c9795cba07df893be" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 38009 } }
I20260811 22:13:43.409143 18192 raft_consensus.cc:374] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:43.409263 18192 raft_consensus.cc:729] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a72b6dfbdd4d4603869c41702ed0af75, State: Initialized, Role: FOLLOWER
I20260811 22:13:43.409214 18191 consensus_queue.cc:260] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eee4251762342c494192fa26ea7a18f" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37387 } } peers { permanent_uuid: "a72b6dfbdd4d4603869c41702ed0af75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 34891 } } peers { permanent_uuid: "3521c69481e4451c9795cba07df893be" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 38009 } }
I20260811 22:13:43.409766 18192 consensus_queue.cc:260] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eee4251762342c494192fa26ea7a18f" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37387 } } peers { permanent_uuid: "a72b6dfbdd4d4603869c41702ed0af75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 34891 } } peers { permanent_uuid: "3521c69481e4451c9795cba07df893be" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 38009 } }
I20260811 22:13:43.409921 18196 log.cc:824] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:43.410523 18171 heartbeater.cc:503] Master 127.28.47.190:46771 was elected leader, sending a full tablet report...
I20260811 22:13:43.410748 18192 ts_tablet_manager.cc:1428] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:13:43.411518 18191 ts_tablet_manager.cc:1428] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:13:43.412289 17869 heartbeater.cc:503] Master 127.28.47.190:46771 was elected leader, sending a full tablet report...
I20260811 22:13:43.412957 18196 tablet_bootstrap.cc:492] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f: No bootstrap required, opened a new log
I20260811 22:13:43.413209 18196 ts_tablet_manager.cc:1397] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:13:43.417236 18196 raft_consensus.cc:348] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eee4251762342c494192fa26ea7a18f" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37387 } } peers { permanent_uuid: "a72b6dfbdd4d4603869c41702ed0af75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 34891 } } peers { permanent_uuid: "3521c69481e4451c9795cba07df893be" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 38009 } }
I20260811 22:13:43.417515 18196 raft_consensus.cc:374] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:43.417604 18196 raft_consensus.cc:729] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1eee4251762342c494192fa26ea7a18f, State: Initialized, Role: FOLLOWER
I20260811 22:13:43.418063 18196 consensus_queue.cc:260] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eee4251762342c494192fa26ea7a18f" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37387 } } peers { permanent_uuid: "a72b6dfbdd4d4603869c41702ed0af75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 34891 } } peers { permanent_uuid: "3521c69481e4451c9795cba07df893be" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 38009 } }
I20260811 22:13:43.418784 18022 heartbeater.cc:503] Master 127.28.47.190:46771 was elected leader, sending a full tablet report...
I20260811 22:13:43.419157 18196 ts_tablet_manager.cc:1428] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
W20260811 22:13:43.432591 17870 tablet.cc:2369] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:13:43.559877 18201 raft_consensus.cc:482] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:13:43.560107 18201 raft_consensus.cc:504] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eee4251762342c494192fa26ea7a18f" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37387 } } peers { permanent_uuid: "a72b6dfbdd4d4603869c41702ed0af75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 34891 } } peers { permanent_uuid: "3521c69481e4451c9795cba07df893be" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 38009 } }
I20260811 22:13:43.561480 18201 leader_election.cc:288] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a72b6dfbdd4d4603869c41702ed0af75 (127.28.47.131:34891), 3521c69481e4451c9795cba07df893be (127.28.47.129:38009)
I20260811 22:13:43.565894 18024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.561746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:38009 (local address 127.28.47.130:52219)
0811 22:13:43.561872 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:43.561891 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:43.562003 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:43.562416 (+ 413us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:43.562420 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:43.562448 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:43.562605 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.562616 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.563600 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.563603 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.564552 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.564562 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.565394 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.565398 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.565504 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.565545 (+ 41us) client_negotiation.cc:769] Sending connection context
0811 22:13:43.565627 (+ 82us) client_negotiation.cc:240] Negotiation successful
0811 22:13:43.565740 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":3}
I20260811 22:13:43.565908 18190 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.561864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:52219 (local address 127.28.47.129:38009)
0811 22:13:43.562028 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:43.562032 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:43.562073 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:13:43.562146 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:43.562148 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:43.562205 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:43.562307 (+ 102us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:43.562846 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.563464 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.564692 (+ 1228us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.565234 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.565350 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.565398 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:43.565663 (+ 265us) server_negotiation.cc:299] Negotiation successful
0811 22:13:43.565760 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:13:43.566162 18188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.561860 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:48127 (local address 127.28.47.131:34891)
0811 22:13:43.561994 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:43.561999 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:43.562386 (+ 387us) server_negotiation.cc:407] Connection header received
0811 22:13:43.562461 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:43.562464 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:43.562537 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:43.562656 (+ 119us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:43.563128 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.563714 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.564930 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.565475 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.565638 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.565685 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:43.565929 (+ 244us) server_negotiation.cc:299] Negotiation successful
0811 22:13:43.566028 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:13:43.566581 18202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.561751 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:34891 (local address 127.28.47.130:48127)
0811 22:13:43.562231 (+ 480us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:43.562247 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:43.562353 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:43.562799 (+ 446us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:43.562803 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:43.562826 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:43.562981 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.562989 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.563887 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.563890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.564698 (+ 808us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:43.564708 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.565648 (+ 940us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.565652 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:43.565786 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.565809 (+ 23us) client_negotiation.cc:769] Sending connection context
0811 22:13:43.565888 (+ 79us) client_negotiation.cc:240] Negotiation successful
0811 22:13:43.566463 (+ 575us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":409,"mutex_wait_us":145,"thread_start_us":147,"threads_started":1}
I20260811 22:13:43.566737 17818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac603fb5b5f54c789e775ed28c36e91f" candidate_uuid: "1eee4251762342c494192fa26ea7a18f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3521c69481e4451c9795cba07df893be" is_pre_election: true
I20260811 22:13:43.567139 17818 raft_consensus.cc:2454] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1eee4251762342c494192fa26ea7a18f in term 0.
I20260811 22:13:43.567322 18117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac603fb5b5f54c789e775ed28c36e91f" candidate_uuid: "1eee4251762342c494192fa26ea7a18f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a72b6dfbdd4d4603869c41702ed0af75" is_pre_election: true
I20260811 22:13:43.567626 18117 raft_consensus.cc:2454] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1eee4251762342c494192fa26ea7a18f in term 0.
I20260811 22:13:43.567785 17889 leader_election.cc:302] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1eee4251762342c494192fa26ea7a18f, 3521c69481e4451c9795cba07df893be; no voters:
I20260811 22:13:43.568127 18201 raft_consensus.cc:2781] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:13:43.568221 18201 raft_consensus.cc:482] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:13:43.568267 18201 raft_consensus.cc:3037] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:43.569437 18201 raft_consensus.cc:504] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eee4251762342c494192fa26ea7a18f" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37387 } } peers { permanent_uuid: "a72b6dfbdd4d4603869c41702ed0af75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 34891 } } peers { permanent_uuid: "3521c69481e4451c9795cba07df893be" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 38009 } }
I20260811 22:13:43.569798 18201 leader_election.cc:288] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [CANDIDATE]: Term 1 election: Requested vote from peers a72b6dfbdd4d4603869c41702ed0af75 (127.28.47.131:34891), 3521c69481e4451c9795cba07df893be (127.28.47.129:38009)
I20260811 22:13:43.570163 18117 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac603fb5b5f54c789e775ed28c36e91f" candidate_uuid: "1eee4251762342c494192fa26ea7a18f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a72b6dfbdd4d4603869c41702ed0af75"
I20260811 22:13:43.570174 17818 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "ac603fb5b5f54c789e775ed28c36e91f" candidate_uuid: "1eee4251762342c494192fa26ea7a18f" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3521c69481e4451c9795cba07df893be"
I20260811 22:13:43.570309 18117 raft_consensus.cc:3037] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:43.570325 17818 raft_consensus.cc:3037] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:43.571512 18117 raft_consensus.cc:2454] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1eee4251762342c494192fa26ea7a18f in term 1.
I20260811 22:13:43.571623 17818 raft_consensus.cc:2454] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1eee4251762342c494192fa26ea7a18f in term 1.
I20260811 22:13:43.571949 17892 leader_election.cc:302] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1eee4251762342c494192fa26ea7a18f, a72b6dfbdd4d4603869c41702ed0af75; no voters:
I20260811 22:13:43.572185 18201 raft_consensus.cc:2781] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:43.572489 18201 raft_consensus.cc:686] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [term 1 LEADER]: Becoming Leader. State: Replica: 1eee4251762342c494192fa26ea7a18f, State: Running, Role: LEADER
I20260811 22:13:43.572853 18201 consensus_queue.cc:237] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eee4251762342c494192fa26ea7a18f" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37387 } } peers { permanent_uuid: "a72b6dfbdd4d4603869c41702ed0af75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 34891 } } peers { permanent_uuid: "3521c69481e4451c9795cba07df893be" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 38009 } }
I20260811 22:13:43.576546 17645 catalog_manager.cc:5495] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f reported cstate change: term changed from 0 to 1, leader changed from <none> to 1eee4251762342c494192fa26ea7a18f (127.28.47.130). New cstate: current_term: 1 leader_uuid: "1eee4251762342c494192fa26ea7a18f" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1eee4251762342c494192fa26ea7a18f" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 37387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "a72b6dfbdd4d4603869c41702ed0af75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 34891 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3521c69481e4451c9795cba07df893be" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 38009 } health_report { overall_health: UNKNOWN } } }
W20260811 22:13:43.586413 18172 tablet.cc:2369] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:13:43.629555 18187 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.624391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35708 (local address 127.28.47.130:37387)
0811 22:13:43.624552 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:43.624556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:43.624570 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:13:43.624653 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:43.624657 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:43.624717 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:43.624854 (+ 137us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:13:43.625124 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.625830 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.627234 (+ 1404us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:43.627528 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:43.627668 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:43.628354 (+ 686us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:13:43.629021 (+ 667us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:13:43.629134 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:43.629323 (+ 189us) server_negotiation.cc:299] Negotiation successful
0811 22:13:43.629394 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":76}
I20260811 22:13:43.636610 17818 raft_consensus.cc:1261] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be [term 1 FOLLOWER]: Refusing update from remote peer 1eee4251762342c494192fa26ea7a18f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:43.636651 18117 raft_consensus.cc:1261] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75 [term 1 FOLLOWER]: Refusing update from remote peer 1eee4251762342c494192fa26ea7a18f: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
W20260811 22:13:43.636847 18023 tablet.cc:2369] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:13:43.637596 18201 consensus_queue.cc:1035] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [LEADER]: Connected to new peer: Peer: permanent_uuid: "a72b6dfbdd4d4603869c41702ed0af75" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 34891 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:43.637907 18203 consensus_queue.cc:1035] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [LEADER]: Connected to new peer: Peer: permanent_uuid: "3521c69481e4451c9795cba07df893be" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 38009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:43.649454 18208 mvcc.cc:205] Tried to move back new op lower bound from 7317448391206694912 to 7317448390957641728. Current Snapshot: MvccSnapshot[applied={T|T < 7317448391206694912}]
I20260811 22:13:43.650194 18207 mvcc.cc:205] Tried to move back new op lower bound from 7317448391206694912 to 7317448390957641728. Current Snapshot: MvccSnapshot[applied={T|T < 7317448391206694912}]
I20260811 22:13:43.658847 17646 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:60312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:43.659024 17646 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:60312:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:43.662063 17646 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9527da75f7d24805843ffc3434e54a33: Sending DeleteTablet for 3 replicas of tablet ac603fb5b5f54c789e775ed28c36e91f
I20260811 22:13:43.662992 17939 tablet_service.cc:1514] Processing DeleteTablet for tablet ac603fb5b5f54c789e775ed28c36e91f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:43 UTC) from {username='slave'} at 127.0.0.1:35692
I20260811 22:13:43.663033 18093 tablet_service.cc:1514] Processing DeleteTablet for tablet ac603fb5b5f54c789e775ed28c36e91f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:43 UTC) from {username='slave'} at 127.0.0.1:53110
I20260811 22:13:43.663612 18216 tablet_replica.cc:330] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f: stopping tablet replica
I20260811 22:13:43.663661 17790 tablet_service.cc:1514] Processing DeleteTablet for tablet ac603fb5b5f54c789e775ed28c36e91f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:43 UTC) from {username='slave'} at 127.0.0.1:44422
I20260811 22:13:43.663987 18216 raft_consensus.cc:2229] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:13:43.664275 18220 tablet_replica.cc:330] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be: stopping tablet replica
I20260811 22:13:43.664556 18216 raft_consensus.cc:2258] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:43.664736 18220 raft_consensus.cc:2229] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:43.664929 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 17718
I20260811 22:13:43.665346 18220 raft_consensus.cc:2258] T ac603fb5b5f54c789e775ed28c36e91f P 3521c69481e4451c9795cba07df893be [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:43.668047 18216 ts_tablet_manager.cc:1905] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:43.671025 18218 tablet_replica.cc:330] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75: stopping tablet replica
I20260811 22:13:43.671180 18216 ts_tablet_manager.cc:1918] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:43.671300 18216 log.cc:1196] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/ac603fb5b5f54c789e775ed28c36e91f
I20260811 22:13:43.671648 18216 ts_tablet_manager.cc:1939] T ac603fb5b5f54c789e775ed28c36e91f P 1eee4251762342c494192fa26ea7a18f: Deleting consensus metadata
I20260811 22:13:43.671648 18218 raft_consensus.cc:2229] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:43.672207 18218 raft_consensus.cc:2258] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:43.673298 17628 catalog_manager.cc:4841] TS 1eee4251762342c494192fa26ea7a18f (127.28.47.130:37387): tablet ac603fb5b5f54c789e775ed28c36e91f (table test-table [id=2fde2321a9574dea85d7ba5172e703cb]) successfully deleted
I20260811 22:13:43.674108 18218 ts_tablet_manager.cc:1905] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:43.676671 18218 ts_tablet_manager.cc:1918] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:43.676873 18218 log.cc:1196] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/ac603fb5b5f54c789e775ed28c36e91f
I20260811 22:13:43.677242 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 17872
I20260811 22:13:43.677258 18218 ts_tablet_manager.cc:1939] T ac603fb5b5f54c789e775ed28c36e91f P a72b6dfbdd4d4603869c41702ed0af75: Deleting consensus metadata
W20260811 22:13:43.677337 17627 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:38009 (error 108)
I20260811 22:13:43.681803 17630 catalog_manager.cc:4841] TS a72b6dfbdd4d4603869c41702ed0af75 (127.28.47.131:34891): tablet ac603fb5b5f54c789e775ed28c36e91f (table test-table [id=2fde2321a9574dea85d7ba5172e703cb]) successfully deleted
I20260811 22:13:43.682492 18184 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:43.682171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:38009 (local address 127.0.0.1:44436)
0811 22:13:43.682325 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:43.682409 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:38009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
W20260811 22:13:43.682674 17627 catalog_manager.cc:4584] TS 3521c69481e4451c9795cba07df893be (127.28.47.129:38009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet ac603fb5b5f54c789e775ed28c36e91f: Network error: Client connection negotiation failed: client connection to 127.28.47.129:38009: connect: Connection refused (error 111)
I20260811 22:13:43.687865 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 18025
I20260811 22:13:43.697408 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 17611
2026-08-11T22:13:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1299 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:13:43 dist-test-slave-vxj2 krb5kdc[18224](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:13:43 dist-test-slave-vxj2 krb5kdc[18224](info): set up 2 sockets
Aug 11 22:13:43 dist-test-slave-vxj2 krb5kdc[18224](info): commencing operation
krb5kdc: starting...
W20260811 22:13:44.841646 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.119s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:13:44 dist-test-slave-vxj2 krb5kdc[18224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:13:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:13:44.980630 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:39089
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:42983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:39089
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:45.101225 18238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:45.101606 18238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:45.101720 18238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:45.106050 18238 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:45.106199 18238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:45.106240 18238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:45.106284 18238 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:45.106321 18238 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:45.112409 18238 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:42983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:39089
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:39089
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:45.113801 18238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:45.115010 18238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:45.124390 18243 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:45.124418 18246 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:45.124562 18238 server_base.cc:1034] running on GCE node
W20260811 22:13:45.124390 18244 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:45.125490 18238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:45.126854 18238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:45.128058 18238 hybrid_clock.cc:648] HybridClock initialized: now 1786486425128018 us; error 54 us; skew 500 ppm
Aug 11 22:13:45 dist-test-slave-vxj2 krb5kdc[18224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:45.131383 18238 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:13:45.132443 18238 webserver.cc:460] Webserver started at http://127.28.47.190:40821/ using document root <none> and password file <none>
I20260811 22:13:45.133141 18238 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:45.133303 18238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:45.133565 18238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:45.135973 18238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "f2dc3800de944d2693d8ba20dcc66cc4"
format_stamp: "Formatted at 2026-08-11 22:13:45 on dist-test-slave-vxj2"
server_key: "b312913e9b563c92fb621e6f1d65a162"
server_key_iv: "2e4510c56852fbe2b2aefa565312a38c"
server_key_version: "encryptionkey@0"
I20260811 22:13:45.136622 18238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "f2dc3800de944d2693d8ba20dcc66cc4"
format_stamp: "Formatted at 2026-08-11 22:13:45 on dist-test-slave-vxj2"
server_key: "b312913e9b563c92fb621e6f1d65a162"
server_key_iv: "2e4510c56852fbe2b2aefa565312a38c"
server_key_version: "encryptionkey@0"
I20260811 22:13:45.140882 18238 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:13:45.143944 18253 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:45.145435 18238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:45.145602 18238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "f2dc3800de944d2693d8ba20dcc66cc4"
format_stamp: "Formatted at 2026-08-11 22:13:45 on dist-test-slave-vxj2"
server_key: "b312913e9b563c92fb621e6f1d65a162"
server_key_iv: "2e4510c56852fbe2b2aefa565312a38c"
server_key_version: "encryptionkey@0"
I20260811 22:13:45.145747 18238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:45.156673 18238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:45.160395 18238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:45.160630 18238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:45.168573 18238 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:39089
I20260811 22:13:45.168605 18311 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:39089 every 8 connection(s)
I20260811 22:13:45.169665 18238 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:45.172724 18312 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:45.178606 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 18238
I20260811 22:13:45.178757 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:13:45.178776 18312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4: Bootstrap starting.
I20260811 22:13:45.179060 28862 external_mini_cluster.cc:1428] Setting key 9938bb14b17c16b8d1483445374f8b48
I20260811 22:13:45.181677 18312 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:45.182474 18312 log.cc:824] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:45.184514 18312 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4: No bootstrap required, opened a new log
Aug 11 22:13:45 dist-test-slave-vxj2 krb5kdc[18224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486424, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:13:45.188328 18312 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2dc3800de944d2693d8ba20dcc66cc4" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 39089 } }
I20260811 22:13:45.188628 18312 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:45.188724 18312 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2dc3800de944d2693d8ba20dcc66cc4, State: Initialized, Role: FOLLOWER
I20260811 22:13:45.189287 18312 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2dc3800de944d2693d8ba20dcc66cc4" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 39089 } }
I20260811 22:13:45.189473 18312 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:45.189559 18312 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:45.189680 18312 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:45.191094 18312 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2dc3800de944d2693d8ba20dcc66cc4" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 39089 } }
I20260811 22:13:45.191532 18312 leader_election.cc:302] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2dc3800de944d2693d8ba20dcc66cc4; no voters:
I20260811 22:13:45.191951 18312 leader_election.cc:288] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:45.192036 18321 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:45.192265 18321 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [term 1 LEADER]: Becoming Leader. State: Replica: f2dc3800de944d2693d8ba20dcc66cc4, State: Running, Role: LEADER
I20260811 22:13:45.192616 18321 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2dc3800de944d2693d8ba20dcc66cc4" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 39089 } }
I20260811 22:13:45.193456 18312 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:45.194443 18323 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2dc3800de944d2693d8ba20dcc66cc4. Latest consensus state: current_term: 1 leader_uuid: "f2dc3800de944d2693d8ba20dcc66cc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2dc3800de944d2693d8ba20dcc66cc4" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 39089 } } }
I20260811 22:13:45.194605 18323 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:45.194955 18322 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2dc3800de944d2693d8ba20dcc66cc4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2dc3800de944d2693d8ba20dcc66cc4" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 39089 } } }
I20260811 22:13:45.195065 18322 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:45.195487 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:45.180661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52008 (local address 127.28.47.190:39089)
0811 22:13:45.181144 (+ 483us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:45.181226 (+ 82us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:45.181268 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:13:45.182043 (+ 775us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:45.182063 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:45.182414 (+ 351us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:45.182805 (+ 391us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:45.183679 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.184732 (+ 1053us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.185690 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.186193 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.186372 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:45.189274 (+ 2902us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:45.189280 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:45.189299 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:45.189341 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:45.191552 (+ 2211us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:45.192151 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:45.192158 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:45.192172 (+ 14us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:45.192363 (+ 191us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:45.192707 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:45.192710 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:45.192712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:45.193314 (+ 602us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:45.193560 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:45.194079 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 22:13:45.194427 (+ 348us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":146,"threads_started":1}
W20260811 22:13:45.199290 18338 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:13:45.199384 18338 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:13:45.199462 18332 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:45.200376 18332 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:45.207297 18332 catalog_manager.cc:1347] Generated new cluster ID: 645a489d132349a394df6b5cf64c26b2
I20260811 22:13:45.207417 18332 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:45.231479 18332 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:45.232959 18332 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:45.242443 18332 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4: Generated new TSK 0
I20260811 22:13:45.243386 18332 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:13:45.284906 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:39089
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:42983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:45.417995 18344 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:45.418368 18344 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:45.418466 18344 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:45.422865 18344 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:45.422976 18344 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:45.423077 18344 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:13:45.429071 18344 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:42983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:39089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18344
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:45.430478 18344 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:45.431695 18344 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:45.442816 18353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:45.442807 18350 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:45.442852 18349 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:45.442952 18344 server_base.cc:1034] running on GCE node
I20260811 22:13:45.443558 18344 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:45.444394 18344 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:45.445604 18344 hybrid_clock.cc:648] HybridClock initialized: now 1786486425445565 us; error 53 us; skew 500 ppm
Aug 11 22:13:45 dist-test-slave-vxj2 krb5kdc[18224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:45.448680 18344 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:13:45.449851 18344 webserver.cc:460] Webserver started at http://127.28.47.129:41825/ using document root <none> and password file <none>
I20260811 22:13:45.450484 18344 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:45.450580 18344 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:45.450778 18344 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:45.452955 18344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "df66c59fbe8048dbad2d41c1027a4155"
format_stamp: "Formatted at 2026-08-11 22:13:45 on dist-test-slave-vxj2"
server_key: "4443f0d4fb0fd699a8530df366898eca"
server_key_iv: "a5b7967591938375daa1dfccd801747f"
server_key_version: "encryptionkey@0"
I20260811 22:13:45.453639 18344 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "df66c59fbe8048dbad2d41c1027a4155"
format_stamp: "Formatted at 2026-08-11 22:13:45 on dist-test-slave-vxj2"
server_key: "4443f0d4fb0fd699a8530df366898eca"
server_key_iv: "a5b7967591938375daa1dfccd801747f"
server_key_version: "encryptionkey@0"
I20260811 22:13:45.457940 18344 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:13:45.460747 18365 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:45.462296 18344 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:45.462459 18344 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "df66c59fbe8048dbad2d41c1027a4155"
format_stamp: "Formatted at 2026-08-11 22:13:45 on dist-test-slave-vxj2"
server_key: "4443f0d4fb0fd699a8530df366898eca"
server_key_iv: "a5b7967591938375daa1dfccd801747f"
server_key_version: "encryptionkey@0"
I20260811 22:13:45.462591 18344 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:45.474713 18344 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:45.478674 18344 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:45.478981 18344 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:45.479703 18344 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:45.480804 18344 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:45.480862 18344 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:45.480917 18344 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:45.480934 18344 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:45.491973 18344 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:35763
I20260811 22:13:45.491997 18498 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:35763 every 8 connection(s)
I20260811 22:13:45.493034 18344 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:13:45.494525 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 18344
I20260811 22:13:45.494791 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:13:45.495134 28862 external_mini_cluster.cc:1428] Setting key 6e69dafed125fcb3827927d94ca3a4e0
Aug 11 22:13:45 dist-test-slave-vxj2 krb5kdc[18224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:13:45.512298 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:45.495350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:53573 (local address 127.28.47.190:39089)
0811 22:13:45.495527 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:45.495532 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:45.497200 (+ 1668us) server_negotiation.cc:407] Connection header received
0811 22:13:45.498747 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:45.498750 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:45.498804 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:45.498935 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:45.500414 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.500970 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.503071 (+ 2101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.503354 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.503463 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:45.507225 (+ 3762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:45.507229 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:45.507231 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:45.507262 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:45.509382 (+ 2120us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:45.510100 (+ 718us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:45.510109 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:45.510111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:45.510201 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:45.510672 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:45.510679 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:45.510684 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:45.511010 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:45.511110 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:45.511987 (+ 877us) server_negotiation.cc:299] Negotiation successful
0811 22:13:45.512133 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:13:45.513545 18501 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:45.495935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:39089 (local address 127.28.47.129:53573)
0811 22:13:45.497000 (+ 1065us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:45.497067 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:45.498428 (+ 1361us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:45.499159 (+ 731us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:45.499167 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:45.499572 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:45.500175 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:45.500218 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.501135 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.501141 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:45.502854 (+ 1713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:45.502875 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.503583 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.503590 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:45.503918 (+ 328us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:45.504770 (+ 852us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:45.504797 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:45.506933 (+ 2136us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:45.509565 (+ 2632us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:45.509573 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:45.509598 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:45.509960 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:45.510309 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:45.510312 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:45.510315 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:45.510515 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:45.511167 (+ 652us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:45.511178 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:45.511655 (+ 477us) client_negotiation.cc:769] Sending connection context
0811 22:13:45.511909 (+ 254us) client_negotiation.cc:240] Negotiation successful
0811 22:13:45.512383 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":789,"thread_start_us":633,"threads_started":1}
I20260811 22:13:45.515027 18499 heartbeater.cc:347] Connected to a master server at 127.28.47.190:39089
I20260811 22:13:45.515319 18499 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:45.516047 18499 heartbeater.cc:511] Master 127.28.47.190:39089 requested a full tablet report, sending...
I20260811 22:13:45.518286 18271 ts_manager.cc:194] Registered new tserver with Master: df66c59fbe8048dbad2d41c1027a4155 (127.28.47.129:35763)
I20260811 22:13:45.520073 18271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:53573
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:13:45.532123 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:39089
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:42983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:45.653096 18504 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:45.653482 18504 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:45.653530 18504 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:45.657780 18504 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:45.657858 18504 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:45.657922 18504 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:13:45.663578 18504 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:42983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:39089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18504
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:45.664700 18504 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:45.665887 18504 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:45.676278 18511 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:45.676285 18510 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:45.676285 18513 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:45.676669 18504 server_base.cc:1034] running on GCE node
I20260811 22:13:45.677119 18504 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:45.677946 18504 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:45.679165 18504 hybrid_clock.cc:648] HybridClock initialized: now 1786486425679142 us; error 32 us; skew 500 ppm
Aug 11 22:13:45 dist-test-slave-vxj2 krb5kdc[18224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:45.682427 18504 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:13:45.683471 18504 webserver.cc:460] Webserver started at http://127.28.47.130:42265/ using document root <none> and password file <none>
I20260811 22:13:45.684105 18504 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:45.684206 18504 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:45.684458 18504 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:45.686801 18504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "be4423dbcae64bcfbeb681f02ced167a"
format_stamp: "Formatted at 2026-08-11 22:13:45 on dist-test-slave-vxj2"
server_key: "aa97cb37a92708a46b8b0c5c9a768e42"
server_key_iv: "10b4d71ea151c8fa45f6aa400a7e6440"
server_key_version: "encryptionkey@0"
I20260811 22:13:45.687443 18504 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "be4423dbcae64bcfbeb681f02ced167a"
format_stamp: "Formatted at 2026-08-11 22:13:45 on dist-test-slave-vxj2"
server_key: "aa97cb37a92708a46b8b0c5c9a768e42"
server_key_iv: "10b4d71ea151c8fa45f6aa400a7e6440"
server_key_version: "encryptionkey@0"
I20260811 22:13:45.691635 18504 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:13:45.694621 18521 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:45.696269 18504 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:13:45.696420 18504 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "be4423dbcae64bcfbeb681f02ced167a"
format_stamp: "Formatted at 2026-08-11 22:13:45 on dist-test-slave-vxj2"
server_key: "aa97cb37a92708a46b8b0c5c9a768e42"
server_key_iv: "10b4d71ea151c8fa45f6aa400a7e6440"
server_key_version: "encryptionkey@0"
I20260811 22:13:45.696544 18504 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:45.719720 18504 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:45.723835 18504 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:45.724090 18504 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:45.724823 18504 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:45.725859 18504 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:45.725911 18504 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:45.725991 18504 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:45.726025 18504 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:45.737079 18504 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:41565
I20260811 22:13:45.737098 18658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:41565 every 8 connection(s)
I20260811 22:13:45.738134 18504 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:13:45.740546 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 18504
I20260811 22:13:45.740658 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:13:45.740953 28862 external_mini_cluster.cc:1428] Setting key 80bde11d830d228e41a12676b05ca468
Aug 11 22:13:45 dist-test-slave-vxj2 krb5kdc[18224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:13:45.754887 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:45.740121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:39969 (local address 127.28.47.190:39089)
0811 22:13:45.740279 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:45.740283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:45.741947 (+ 1664us) server_negotiation.cc:407] Connection header received
0811 22:13:45.743207 (+ 1260us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:45.743212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:45.743275 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:45.743373 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:45.744852 (+ 1479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.745472 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.746748 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.746985 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.747101 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:45.750265 (+ 3164us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:45.750269 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:45.750271 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:45.750300 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:45.752178 (+ 1878us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:45.752858 (+ 680us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:45.752861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:45.752863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:45.752962 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:45.753528 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:45.753531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:45.753533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:45.753772 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:45.753901 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:45.754557 (+ 656us) server_negotiation.cc:299] Negotiation successful
0811 22:13:45.754709 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
I20260811 22:13:45.756054 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:45.740546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:39089 (local address 127.28.47.130:39969)
0811 22:13:45.741731 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:45.741809 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:45.742976 (+ 1167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:45.743566 (+ 590us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:45.743574 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:45.744059 (+ 485us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:45.744687 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:45.744704 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.745618 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.745622 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:45.746594 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:45.746605 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.747174 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.747178 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:45.747373 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:45.748139 (+ 766us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:45.748157 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:45.750043 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:45.752358 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:45.752366 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:45.752387 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:45.752710 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:45.753067 (+ 357us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:45.753070 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:45.753072 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:45.753390 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:45.753965 (+ 575us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:45.753971 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:45.754274 (+ 303us) client_negotiation.cc:769] Sending connection context
0811 22:13:45.754492 (+ 218us) client_negotiation.cc:240] Negotiation successful
0811 22:13:45.754905 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":915,"thread_start_us":693,"threads_started":1}
I20260811 22:13:45.757819 18659 heartbeater.cc:347] Connected to a master server at 127.28.47.190:39089
I20260811 22:13:45.758127 18659 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:45.758728 18659 heartbeater.cc:511] Master 127.28.47.190:39089 requested a full tablet report, sending...
I20260811 22:13:45.760033 18271 ts_manager.cc:194] Registered new tserver with Master: be4423dbcae64bcfbeb681f02ced167a (127.28.47.130:41565)
I20260811 22:13:45.760666 18271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:39969
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:13:45.773418 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:39089
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:42983
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:45.892285 18667 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:45.892644 18667 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:45.892722 18667 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:45.897320 18667 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:45.897444 18667 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:45.897541 18667 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:13:45.903501 18667 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:42983
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:39089
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18667
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:45.904731 18667 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:45.905908 18667 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:45.916808 18674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:45.916826 18672 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:45.916862 18667 server_base.cc:1034] running on GCE node
W20260811 22:13:45.916976 18679 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:45.917588 18667 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:45.918397 18667 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:45.919610 18667 hybrid_clock.cc:648] HybridClock initialized: now 1786486425919547 us; error 81 us; skew 500 ppm
Aug 11 22:13:45 dist-test-slave-vxj2 krb5kdc[18224](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:45.923135 18667 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:13:45.924259 18667 webserver.cc:460] Webserver started at http://127.28.47.131:42275/ using document root <none> and password file <none>
I20260811 22:13:45.924924 18667 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:45.925004 18667 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:45.925304 18667 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:45.927639 18667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "55de768518404f0ca2299863f902a8b0"
format_stamp: "Formatted at 2026-08-11 22:13:45 on dist-test-slave-vxj2"
server_key: "8cd811c4722ec03865a1e5e26a8bf874"
server_key_iv: "18d7e085c9faa3cdb5559dd4b7504814"
server_key_version: "encryptionkey@0"
I20260811 22:13:45.928299 18667 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "55de768518404f0ca2299863f902a8b0"
format_stamp: "Formatted at 2026-08-11 22:13:45 on dist-test-slave-vxj2"
server_key: "8cd811c4722ec03865a1e5e26a8bf874"
server_key_iv: "18d7e085c9faa3cdb5559dd4b7504814"
server_key_version: "encryptionkey@0"
I20260811 22:13:45.932636 18667 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260811 22:13:45.935573 18686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:45.937144 18667 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:13:45.937356 18667 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "55de768518404f0ca2299863f902a8b0"
format_stamp: "Formatted at 2026-08-11 22:13:45 on dist-test-slave-vxj2"
server_key: "8cd811c4722ec03865a1e5e26a8bf874"
server_key_iv: "18d7e085c9faa3cdb5559dd4b7504814"
server_key_version: "encryptionkey@0"
I20260811 22:13:45.937489 18667 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:45.953374 18667 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:45.957078 18667 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:45.957401 18667 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:45.958110 18667 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:45.959030 18667 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:45.959079 18667 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:45.959147 18667 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:45.959182 18667 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:45.969952 18667 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:35269
I20260811 22:13:45.969975 18816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:35269 every 8 connection(s)
I20260811 22:13:45.970935 18667 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:13:45.971763 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 18667
I20260811 22:13:45.971875 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:13:45.972282 28862 external_mini_cluster.cc:1428] Setting key a6f23bee5804ea124f8bcfc840a1d25e
Aug 11 22:13:45 dist-test-slave-vxj2 krb5kdc[18224](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486425, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:13:45.985440 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:45.973145 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:36775 (local address 127.28.47.190:39089)
0811 22:13:45.973304 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:45.973309 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:45.974304 (+ 995us) server_negotiation.cc:407] Connection header received
0811 22:13:45.975749 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:45.975753 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:45.975809 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:45.975880 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:45.977017 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.977583 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.978730 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.978974 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.979064 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:45.981871 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:45.981874 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:45.981876 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:45.981904 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:45.983320 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:45.983860 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:45.983863 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:45.983864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:45.983944 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:45.984252 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:45.984254 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:45.984255 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:45.984442 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:45.984518 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:45.985077 (+ 559us) server_negotiation.cc:299] Negotiation successful
0811 22:13:45.985237 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260811 22:13:45.986421 18819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:45.973651 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:39089 (local address 127.28.47.131:36775)
0811 22:13:45.974156 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:45.974211 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:45.975546 (+ 1335us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:45.976012 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:45.976018 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:45.976360 (+ 342us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:45.976856 (+ 496us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:45.976869 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.977711 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.977714 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:45.978605 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:45.978615 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:45.979114 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:45.979117 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:45.979252 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:45.979913 (+ 661us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:45.979929 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:45.981646 (+ 1717us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:45.983439 (+ 1793us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:45.983444 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:45.983459 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:45.983729 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:45.984030 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:45.984032 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:45.984033 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:45.984172 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:45.984531 (+ 359us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:45.984535 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:45.984792 (+ 257us) client_negotiation.cc:769] Sending connection context
0811 22:13:45.984964 (+ 172us) client_negotiation.cc:240] Negotiation successful
0811 22:13:45.985356 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":136,"threads_started":1}
I20260811 22:13:45.987632 18817 heartbeater.cc:347] Connected to a master server at 127.28.47.190:39089
I20260811 22:13:45.987871 18817 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:45.988348 18817 heartbeater.cc:511] Master 127.28.47.190:39089 requested a full tablet report, sending...
I20260811 22:13:45.989380 18271 ts_manager.cc:194] Registered new tserver with Master: 55de768518404f0ca2299863f902a8b0 (127.28.47.131:35269)
I20260811 22:13:45.989957 18271 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:36775
I20260811 22:13:45.999228 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:13:46.010140 18317 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.001589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52016 (local address 127.28.47.190:39089)
0811 22:13:46.001786 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:46.001791 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:46.001962 (+ 171us) server_negotiation.cc:407] Connection header received
0811 22:13:46.002123 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:46.002126 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:46.002176 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:46.002289 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:46.003077 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.003620 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.004464 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.004702 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.004818 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.006560 (+ 1742us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:46.006564 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:46.006567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:46.006594 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:46.008119 (+ 1525us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:46.008654 (+ 535us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:46.008658 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:46.008659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:46.008747 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:46.009087 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:46.009091 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:46.009092 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:46.009395 (+ 303us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:46.009569 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:46.009861 (+ 292us) server_negotiation.cc:299] Negotiation successful
0811 22:13:46.009964 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260811 22:13:46.013695 18271 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52016:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:13:46.017315 18271 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:13:46.034343 18835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.028470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47554 (local address 127.28.47.131:35269)
0811 22:13:46.029117 (+ 647us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:46.029128 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:46.029411 (+ 283us) server_negotiation.cc:407] Connection header received
0811 22:13:46.029546 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:46.029552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:46.029735 (+ 183us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:46.029904 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:46.030092 (+ 188us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.031185 (+ 1093us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.032681 (+ 1496us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.033543 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.033677 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.033847 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:46.034050 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 22:13:46.034160 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":553,"thread_start_us":113,"threads_started":1}
I20260811 22:13:46.034319 18831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.028032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:35269 (local address 127.0.0.1:47554)
0811 22:13:46.029287 (+ 1255us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:46.029301 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:46.029420 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:46.029897 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:46.029900 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:46.029916 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:46.029997 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.030002 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.031360 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.031364 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.032522 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.032534 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.033682 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.033686 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.033805 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.033864 (+ 59us) client_negotiation.cc:769] Sending connection context
0811 22:13:46.034011 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 22:13:46.034168 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1192,"mutex_wait_us":73,"thread_start_us":96,"threads_started":1}
I20260811 22:13:46.034572 18832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.027543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:41565 (local address 127.0.0.1:59886)
0811 22:13:46.028380 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:46.028439 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:46.028581 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:46.029490 (+ 909us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:46.029498 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:46.029524 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:46.029705 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.029713 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.030963 (+ 1250us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.030970 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.032575 (+ 1605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.032583 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.033985 (+ 1402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.033990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.034168 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.034211 (+ 43us) client_negotiation.cc:769] Sending connection context
0811 22:13:46.034286 (+ 75us) client_negotiation.cc:240] Negotiation successful
0811 22:13:46.034380 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":648,"mutex_wait_us":151,"thread_start_us":126,"threads_started":1}
I20260811 22:13:46.034998 18834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.027779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:35763 (local address 127.0.0.1:43928)
0811 22:13:46.028939 (+ 1160us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:46.028956 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:46.029056 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:46.029672 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:46.029674 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:46.029689 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:46.029766 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.029772 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.031261 (+ 1489us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.031264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.032562 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.032576 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.033985 (+ 1409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.033990 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.034171 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.034215 (+ 44us) client_negotiation.cc:769] Sending connection context
0811 22:13:46.034286 (+ 71us) client_negotiation.cc:240] Negotiation successful
0811 22:13:46.034879 (+ 593us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1091,"mutex_wait_us":664,"thread_start_us":168,"threads_started":1}
I20260811 22:13:46.035090 18836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.028489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43928 (local address 127.28.47.129:35763)
0811 22:13:46.028905 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:46.028913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:46.029198 (+ 285us) server_negotiation.cc:407] Connection header received
0811 22:13:46.029298 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:46.029302 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:46.029518 (+ 216us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:46.029673 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:46.029892 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.031119 (+ 1227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.032722 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.033653 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.033766 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.033918 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:46.034371 (+ 453us) server_negotiation.cc:299] Negotiation successful
0811 22:13:46.034938 (+ 567us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":136,"threads_started":1}
I20260811 22:13:46.035511 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.027749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59886 (local address 127.28.47.130:41565)
0811 22:13:46.028909 (+ 1160us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:46.028921 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:46.028948 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:13:46.029034 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:46.029041 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:46.029306 (+ 265us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:46.029869 (+ 563us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:46.030055 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.030812 (+ 757us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.032722 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.033653 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.033766 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.033913 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:46.035243 (+ 1330us) server_negotiation.cc:299] Negotiation successful
0811 22:13:46.035354 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1036,"thread_start_us":184,"threads_started":1}
I20260811 22:13:46.038100 18581 tablet_service.cc:1467] Processing CreateTablet for tablet 93db7dd708b6464cb7b27b7379bd80af (DEFAULT_TABLE table=test-table [id=935601c781dd46dd9f422270e0ad6311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:46.038058 18418 tablet_service.cc:1467] Processing CreateTablet for tablet 93db7dd708b6464cb7b27b7379bd80af (DEFAULT_TABLE table=test-table [id=935601c781dd46dd9f422270e0ad6311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:46.038064 18741 tablet_service.cc:1467] Processing CreateTablet for tablet 93db7dd708b6464cb7b27b7379bd80af (DEFAULT_TABLE table=test-table [id=935601c781dd46dd9f422270e0ad6311]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:46.039142 18581 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93db7dd708b6464cb7b27b7379bd80af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:46.039177 18741 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93db7dd708b6464cb7b27b7379bd80af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:46.039177 18418 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93db7dd708b6464cb7b27b7379bd80af. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:46.046674 18837 tablet_bootstrap.cc:492] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a: Bootstrap starting.
I20260811 22:13:46.047559 18838 tablet_bootstrap.cc:492] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155: Bootstrap starting.
I20260811 22:13:46.048141 18839 tablet_bootstrap.cc:492] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0: Bootstrap starting.
I20260811 22:13:46.049894 18837 tablet_bootstrap.cc:654] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:46.049963 18838 tablet_bootstrap.cc:654] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:46.050532 18839 tablet_bootstrap.cc:654] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:46.051021 18838 log.cc:824] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:46.051396 18837 log.cc:824] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:46.051824 18839 log.cc:824] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:46.053308 18838 tablet_bootstrap.cc:492] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155: No bootstrap required, opened a new log
I20260811 22:13:46.053336 18837 tablet_bootstrap.cc:492] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a: No bootstrap required, opened a new log
I20260811 22:13:46.053498 18838 ts_tablet_manager.cc:1397] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:13:46.053560 18837 ts_tablet_manager.cc:1397] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:13:46.055020 18839 tablet_bootstrap.cc:492] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0: No bootstrap required, opened a new log
I20260811 22:13:46.055217 18839 ts_tablet_manager.cc:1397] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260811 22:13:46.057061 18837 raft_consensus.cc:348] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4423dbcae64bcfbeb681f02ced167a" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41565 } } peers { permanent_uuid: "df66c59fbe8048dbad2d41c1027a4155" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 35763 } } peers { permanent_uuid: "55de768518404f0ca2299863f902a8b0" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35269 } }
I20260811 22:13:46.057447 18837 raft_consensus.cc:374] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:46.057511 18837 raft_consensus.cc:729] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: be4423dbcae64bcfbeb681f02ced167a, State: Initialized, Role: FOLLOWER
I20260811 22:13:46.058024 18837 consensus_queue.cc:260] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4423dbcae64bcfbeb681f02ced167a" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41565 } } peers { permanent_uuid: "df66c59fbe8048dbad2d41c1027a4155" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 35763 } } peers { permanent_uuid: "55de768518404f0ca2299863f902a8b0" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35269 } }
I20260811 22:13:46.058214 18838 raft_consensus.cc:348] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4423dbcae64bcfbeb681f02ced167a" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41565 } } peers { permanent_uuid: "df66c59fbe8048dbad2d41c1027a4155" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 35763 } } peers { permanent_uuid: "55de768518404f0ca2299863f902a8b0" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35269 } }
I20260811 22:13:46.058496 18838 raft_consensus.cc:374] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:46.058570 18838 raft_consensus.cc:729] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df66c59fbe8048dbad2d41c1027a4155, State: Initialized, Role: FOLLOWER
I20260811 22:13:46.058933 18659 heartbeater.cc:503] Master 127.28.47.190:39089 was elected leader, sending a full tablet report...
I20260811 22:13:46.059062 18838 consensus_queue.cc:260] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4423dbcae64bcfbeb681f02ced167a" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41565 } } peers { permanent_uuid: "df66c59fbe8048dbad2d41c1027a4155" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 35763 } } peers { permanent_uuid: "55de768518404f0ca2299863f902a8b0" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35269 } }
I20260811 22:13:46.059163 18837 ts_tablet_manager.cc:1428] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:13:46.059839 18499 heartbeater.cc:503] Master 127.28.47.190:39089 was elected leader, sending a full tablet report...
I20260811 22:13:46.059852 18839 raft_consensus.cc:348] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4423dbcae64bcfbeb681f02ced167a" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41565 } } peers { permanent_uuid: "df66c59fbe8048dbad2d41c1027a4155" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 35763 } } peers { permanent_uuid: "55de768518404f0ca2299863f902a8b0" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35269 } }
I20260811 22:13:46.060083 18838 ts_tablet_manager.cc:1428] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260811 22:13:46.060096 18839 raft_consensus.cc:374] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:46.060144 18839 raft_consensus.cc:729] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 55de768518404f0ca2299863f902a8b0, State: Initialized, Role: FOLLOWER
I20260811 22:13:46.060571 18839 consensus_queue.cc:260] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4423dbcae64bcfbeb681f02ced167a" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41565 } } peers { permanent_uuid: "df66c59fbe8048dbad2d41c1027a4155" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 35763 } } peers { permanent_uuid: "55de768518404f0ca2299863f902a8b0" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35269 } }
I20260811 22:13:46.061282 18817 heartbeater.cc:503] Master 127.28.47.190:39089 was elected leader, sending a full tablet report...
I20260811 22:13:46.061555 18839 ts_tablet_manager.cc:1428] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:13:46.103616 18848 raft_consensus.cc:482] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:13:46.103905 18848 raft_consensus.cc:504] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4423dbcae64bcfbeb681f02ced167a" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41565 } } peers { permanent_uuid: "df66c59fbe8048dbad2d41c1027a4155" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 35763 } } peers { permanent_uuid: "55de768518404f0ca2299863f902a8b0" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35269 } }
I20260811 22:13:46.105293 18848 leader_election.cc:288] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers df66c59fbe8048dbad2d41c1027a4155 (127.28.47.129:35763), 55de768518404f0ca2299863f902a8b0 (127.28.47.131:35269)
I20260811 22:13:46.109143 18661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.105557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:35269 (local address 127.28.47.130:44827)
0811 22:13:46.105732 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:46.105749 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:46.105860 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:46.106166 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:46.106169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:46.106194 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:46.106337 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.106344 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.107134 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.107140 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.107950 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.107960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.108662 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.108666 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.108778 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.108810 (+ 32us) client_negotiation.cc:769] Sending connection context
0811 22:13:46.108904 (+ 94us) client_negotiation.cc:240] Negotiation successful
0811 22:13:46.108989 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 22:13:46.109143 18835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.105731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:44827 (local address 127.28.47.131:35269)
0811 22:13:46.105893 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:46.105896 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:46.105911 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:13:46.105992 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:46.105995 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:46.106041 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:46.106151 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:46.106498 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.107011 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.108094 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.108527 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.108668 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.108730 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:46.108929 (+ 199us) server_negotiation.cc:299] Negotiation successful
0811 22:13:46.109014 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:13:46.109760 18853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.105560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:35763 (local address 127.28.47.130:56489)
0811 22:13:46.106268 (+ 708us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:46.106279 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:46.106402 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:46.106682 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:46.106684 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:46.106701 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:46.106836 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.106842 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.107579 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.107583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.108388 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.108398 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.109104 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.109108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.109406 (+ 298us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.109435 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:13:46.109515 (+ 80us) client_negotiation.cc:240] Negotiation successful
0811 22:13:46.109595 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":649,"mutex_wait_us":287,"thread_start_us":192,"threads_started":1}
I20260811 22:13:46.109797 18836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.105690 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:56489 (local address 127.28.47.129:35763)
0811 22:13:46.105825 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:46.105829 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:46.106374 (+ 545us) server_negotiation.cc:407] Connection header received
0811 22:13:46.106529 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:46.106533 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:46.106586 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:46.106669 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:46.106955 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.107462 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.108542 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.108974 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.109109 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.109163 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:46.109562 (+ 399us) server_negotiation.cc:299] Negotiation successful
0811 22:13:46.109656 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:13:46.110131 18764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93db7dd708b6464cb7b27b7379bd80af" candidate_uuid: "be4423dbcae64bcfbeb681f02ced167a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55de768518404f0ca2299863f902a8b0" is_pre_election: true
I20260811 22:13:46.110433 18440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93db7dd708b6464cb7b27b7379bd80af" candidate_uuid: "be4423dbcae64bcfbeb681f02ced167a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df66c59fbe8048dbad2d41c1027a4155" is_pre_election: true
I20260811 22:13:46.110522 18764 raft_consensus.cc:2454] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be4423dbcae64bcfbeb681f02ced167a in term 0.
I20260811 22:13:46.110774 18440 raft_consensus.cc:2454] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate be4423dbcae64bcfbeb681f02ced167a in term 0.
I20260811 22:13:46.111272 18522 leader_election.cc:302] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 55de768518404f0ca2299863f902a8b0, be4423dbcae64bcfbeb681f02ced167a, df66c59fbe8048dbad2d41c1027a4155; no voters:
I20260811 22:13:46.111632 18848 raft_consensus.cc:2781] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:13:46.111780 18848 raft_consensus.cc:482] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:13:46.111848 18848 raft_consensus.cc:3037] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:46.113435 18848 raft_consensus.cc:504] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4423dbcae64bcfbeb681f02ced167a" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41565 } } peers { permanent_uuid: "df66c59fbe8048dbad2d41c1027a4155" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 35763 } } peers { permanent_uuid: "55de768518404f0ca2299863f902a8b0" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35269 } }
I20260811 22:13:46.113854 18848 leader_election.cc:288] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [CANDIDATE]: Term 1 election: Requested vote from peers df66c59fbe8048dbad2d41c1027a4155 (127.28.47.129:35763), 55de768518404f0ca2299863f902a8b0 (127.28.47.131:35269)
I20260811 22:13:46.114336 18440 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93db7dd708b6464cb7b27b7379bd80af" candidate_uuid: "be4423dbcae64bcfbeb681f02ced167a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "df66c59fbe8048dbad2d41c1027a4155"
I20260811 22:13:46.114496 18764 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93db7dd708b6464cb7b27b7379bd80af" candidate_uuid: "be4423dbcae64bcfbeb681f02ced167a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "55de768518404f0ca2299863f902a8b0"
I20260811 22:13:46.114626 18440 raft_consensus.cc:3037] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:46.114647 18764 raft_consensus.cc:3037] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:46.115837 18764 raft_consensus.cc:2454] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be4423dbcae64bcfbeb681f02ced167a in term 1.
I20260811 22:13:46.115887 18440 raft_consensus.cc:2454] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate be4423dbcae64bcfbeb681f02ced167a in term 1.
I20260811 22:13:46.116262 18527 leader_election.cc:302] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 55de768518404f0ca2299863f902a8b0, be4423dbcae64bcfbeb681f02ced167a; no voters:
I20260811 22:13:46.116482 18848 raft_consensus.cc:2781] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:46.116757 18848 raft_consensus.cc:686] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [term 1 LEADER]: Becoming Leader. State: Replica: be4423dbcae64bcfbeb681f02ced167a, State: Running, Role: LEADER
I20260811 22:13:46.117203 18848 consensus_queue.cc:237] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4423dbcae64bcfbeb681f02ced167a" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41565 } } peers { permanent_uuid: "df66c59fbe8048dbad2d41c1027a4155" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 35763 } } peers { permanent_uuid: "55de768518404f0ca2299863f902a8b0" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35269 } }
I20260811 22:13:46.121078 18271 catalog_manager.cc:5495] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a reported cstate change: term changed from 0 to 1, leader changed from <none> to be4423dbcae64bcfbeb681f02ced167a (127.28.47.130). New cstate: current_term: 1 leader_uuid: "be4423dbcae64bcfbeb681f02ced167a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "be4423dbcae64bcfbeb681f02ced167a" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 41565 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "df66c59fbe8048dbad2d41c1027a4155" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 35763 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "55de768518404f0ca2299863f902a8b0" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35269 } health_report { overall_health: UNKNOWN } } }
I20260811 22:13:46.141914 18833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.138160 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59892 (local address 127.28.47.130:41565)
0811 22:13:46.138341 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:46.138345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:46.138362 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:13:46.138441 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:46.138443 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:46.138500 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:46.138619 (+ 119us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:13:46.138834 (+ 215us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.139353 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.140167 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.140402 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.140490 (+ 88us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.140751 (+ 261us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:13:46.141387 (+ 636us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:13:46.141514 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:46.141635 (+ 121us) server_negotiation.cc:299] Negotiation successful
0811 22:13:46.141705 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260811 22:13:46.148965 18440 raft_consensus.cc:1261] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155 [term 1 FOLLOWER]: Refusing update from remote peer be4423dbcae64bcfbeb681f02ced167a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:46.148958 18764 raft_consensus.cc:1261] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0 [term 1 FOLLOWER]: Refusing update from remote peer be4423dbcae64bcfbeb681f02ced167a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:46.150115 18848 consensus_queue.cc:1035] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [LEADER]: Connected to new peer: Peer: permanent_uuid: "df66c59fbe8048dbad2d41c1027a4155" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 35763 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:46.150435 18855 consensus_queue.cc:1035] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [LEADER]: Connected to new peer: Peer: permanent_uuid: "55de768518404f0ca2299863f902a8b0" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 35269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:46.169100 18270 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:46.169356 18270 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52016:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:46.171823 18270 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f2dc3800de944d2693d8ba20dcc66cc4: Sending DeleteTablet for 3 replicas of tablet 93db7dd708b6464cb7b27b7379bd80af
I20260811 22:13:46.172674 18581 tablet_service.cc:1514] Processing DeleteTablet for tablet 93db7dd708b6464cb7b27b7379bd80af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:46 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:59886
I20260811 22:13:46.172845 18741 tablet_service.cc:1514] Processing DeleteTablet for tablet 93db7dd708b6464cb7b27b7379bd80af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:46 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:47554
I20260811 22:13:46.173280 18418 tablet_service.cc:1514] Processing DeleteTablet for tablet 93db7dd708b6464cb7b27b7379bd80af with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:46 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:43928
I20260811 22:13:46.173766 18871 tablet_replica.cc:330] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155: stopping tablet replica
I20260811 22:13:46.174168 18871 raft_consensus.cc:2229] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:46.174561 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 18344
I20260811 22:13:46.174695 18871 raft_consensus.cc:2258] T 93db7dd708b6464cb7b27b7379bd80af P df66c59fbe8048dbad2d41c1027a4155 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:46.180934 18870 tablet_replica.cc:330] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0: stopping tablet replica
I20260811 22:13:46.181422 18870 raft_consensus.cc:2229] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:46.181834 18870 raft_consensus.cc:2258] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:46.183677 18870 ts_tablet_manager.cc:1905] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:46.185344 18869 tablet_replica.cc:330] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a: stopping tablet replica
I20260811 22:13:46.185711 18869 raft_consensus.cc:2229] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:13:46.186204 18869 raft_consensus.cc:2258] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:46.187391 18870 ts_tablet_manager.cc:1918] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:46.187517 18870 log.cc:1196] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/93db7dd708b6464cb7b27b7379bd80af
I20260811 22:13:46.187716 18869 ts_tablet_manager.cc:1905] T 93db7dd708b6464cb7b27b7379bd80af P be4423dbcae64bcfbeb681f02ced167a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:46.187870 18870 ts_tablet_manager.cc:1939] T 93db7dd708b6464cb7b27b7379bd80af P 55de768518404f0ca2299863f902a8b0: Deleting consensus metadata
I20260811 22:13:46.189517 18257 catalog_manager.cc:4841] TS 55de768518404f0ca2299863f902a8b0 (127.28.47.131:35269): tablet 93db7dd708b6464cb7b27b7379bd80af (table test-table [id=935601c781dd46dd9f422270e0ad6311]) successfully deleted
W20260811 22:13:46.189863 18254 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:35763 (error 108)
I20260811 22:13:46.190863 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 18504
I20260811 22:13:46.190979 18834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.190617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:35763 (local address 127.0.0.1:43944)
0811 22:13:46.190794 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:46.190885 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:35763: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260811 22:13:46.192516 18254 catalog_manager.cc:4584] TS df66c59fbe8048dbad2d41c1027a4155 (127.28.47.129:35763): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 93db7dd708b6464cb7b27b7379bd80af: Network error: Client connection negotiation failed: client connection to 127.28.47.129:35763: connect: Connection refused (error 111)
I20260811 22:13:46.201211 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 18667
I20260811 22:13:46.203209 18834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.201497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:41565 (local address 127.0.0.1:59900)
0811 22:13:46.203012 (+ 1515us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:46.203090 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.130:41565: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":1385}
I20260811 22:13:46.210186 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 18238
2026-08-11T22:13:46Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2517 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-11T22:13:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:46Z Disabled control of system clock
I20260811 22:13:46.250140 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:42547
--webserver_interface=127.28.47.190
--webserver_port=0
--builtin_ntp_servers=127.28.47.148:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:42547
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:13:46.369706 18876 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:46.369992 18876 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:46.370028 18876 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:46.374122 18876 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:46.374194 18876 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:46.374210 18876 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:46.374228 18876 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:46.374244 18876 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:46.379335 18876 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:42547
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:42547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18876
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:46.380700 18876 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:46.382311 18876 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:46.391793 18882 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:46.391798 18884 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:46.391886 18876 server_base.cc:1034] running on GCE node
W20260811 22:13:46.392055 18881 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:46.392678 18876 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:46.393975 18876 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:46.395169 18876 hybrid_clock.cc:648] HybridClock initialized: now 1786486426395154 us; error 31 us; skew 500 ppm
I20260811 22:13:46.397280 18876 webserver.cc:460] Webserver started at http://127.28.47.190:44369/ using document root <none> and password file <none>
I20260811 22:13:46.397946 18876 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:46.398011 18876 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:46.398241 18876 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:46.400475 18876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "0ed8a73a91304c38af4b30bcad8f1083"
format_stamp: "Formatted at 2026-08-11 22:13:46 on dist-test-slave-vxj2"
server_key: "a8dc8cc3366659e64f688adc3973aaf0"
server_key_iv: "d4b30e3db74f5c226e7b14a9093f241b"
server_key_version: "encryptionkey@0"
I20260811 22:13:46.401069 18876 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "0ed8a73a91304c38af4b30bcad8f1083"
format_stamp: "Formatted at 2026-08-11 22:13:46 on dist-test-slave-vxj2"
server_key: "a8dc8cc3366659e64f688adc3973aaf0"
server_key_iv: "d4b30e3db74f5c226e7b14a9093f241b"
server_key_version: "encryptionkey@0"
I20260811 22:13:46.404860 18876 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:46.407459 18890 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:46.408843 18876 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:13:46.409009 18876 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "0ed8a73a91304c38af4b30bcad8f1083"
format_stamp: "Formatted at 2026-08-11 22:13:46 on dist-test-slave-vxj2"
server_key: "a8dc8cc3366659e64f688adc3973aaf0"
server_key_iv: "d4b30e3db74f5c226e7b14a9093f241b"
server_key_version: "encryptionkey@0"
I20260811 22:13:46.409138 18876 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:46.430066 18876 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:46.430883 18876 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:46.431111 18876 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:46.438714 18876 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:42547
I20260811 22:13:46.438741 18953 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:42547 every 8 connection(s)
I20260811 22:13:46.439779 18876 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:46.443158 18954 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:46.448016 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 18876
I20260811 22:13:46.448146 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:13:46.448451 28862 external_mini_cluster.cc:1428] Setting key 82f6a6e91c4c73cc6542a0f6135980da
I20260811 22:13:46.449685 18954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083: Bootstrap starting.
I20260811 22:13:46.452082 18954 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:46.452915 18954 log.cc:824] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:46.455273 18954 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083: No bootstrap required, opened a new log
I20260811 22:13:46.458196 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.450020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35128 (local address 127.28.47.190:42547)
0811 22:13:46.450679 (+ 659us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:46.450696 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:46.450732 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:13:46.451374 (+ 642us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:46.451390 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:46.451785 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:46.452061 (+ 276us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:46.452494 (+ 433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.453745 (+ 1251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.454946 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.455460 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.455575 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.456445 (+ 870us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:46.456449 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:46.456462 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:46.456834 (+ 372us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:46.456888 (+ 54us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:46.456908 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:46.456989 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:46.457196 (+ 207us) server_negotiation.cc:299] Negotiation successful
0811 22:13:46.457504 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":135,"threads_started":1}
I20260811 22:13:46.459429 18954 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ed8a73a91304c38af4b30bcad8f1083" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42547 } }
I20260811 22:13:46.459695 18954 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:46.459764 18954 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0ed8a73a91304c38af4b30bcad8f1083, State: Initialized, Role: FOLLOWER
I20260811 22:13:46.460225 18954 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ed8a73a91304c38af4b30bcad8f1083" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42547 } }
I20260811 22:13:46.460400 18954 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:46.460474 18954 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:46.460604 18954 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:46.461583 18954 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ed8a73a91304c38af4b30bcad8f1083" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42547 } }
I20260811 22:13:46.461921 18954 leader_election.cc:302] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0ed8a73a91304c38af4b30bcad8f1083; no voters:
I20260811 22:13:46.462230 18954 leader_election.cc:288] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:46.462317 18959 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:46.462587 18959 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [term 1 LEADER]: Becoming Leader. State: Replica: 0ed8a73a91304c38af4b30bcad8f1083, State: Running, Role: LEADER
I20260811 22:13:46.462908 18959 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ed8a73a91304c38af4b30bcad8f1083" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42547 } }
I20260811 22:13:46.463505 18954 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:46.464623 18961 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0ed8a73a91304c38af4b30bcad8f1083. Latest consensus state: current_term: 1 leader_uuid: "0ed8a73a91304c38af4b30bcad8f1083" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ed8a73a91304c38af4b30bcad8f1083" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42547 } } }
I20260811 22:13:46.464771 18961 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:46.465176 18960 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0ed8a73a91304c38af4b30bcad8f1083" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0ed8a73a91304c38af4b30bcad8f1083" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42547 } } }
I20260811 22:13:46.465368 18960 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:46.466082 18970 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:46.468912 18970 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:46.475808 18970 catalog_manager.cc:1347] Generated new cluster ID: 6e5396804eb24e26971d98059d237386
I20260811 22:13:46.475910 18970 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:46.483309 18970 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:46.484622 18970 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:46.497102 18970 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083: Generated new TSK 0
I20260811 22:13:46.497999 18970 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:13:46.503170 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42547
--builtin_ntp_servers=127.28.47.148:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:13:46.624267 18981 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:46.624590 18981 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:46.624634 18981 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:46.629091 18981 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:46.629209 18981 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:46.629271 18981 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:13:46.634580 18981 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18981
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:46.635864 18981 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:46.637493 18981 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260811 22:13:46.648134 18981 server_base.cc:1034] running on GCE node
W20260811 22:13:46.648069 18987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:46.648072 18988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:46.648095 18990 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:46.648813 18981 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:46.649606 18981 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:46.650848 18981 hybrid_clock.cc:648] HybridClock initialized: now 1786486426650814 us; error 45 us; skew 500 ppm
I20260811 22:13:46.653079 18981 webserver.cc:460] Webserver started at http://127.28.47.129:33071/ using document root <none> and password file <none>
I20260811 22:13:46.653761 18981 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:46.653874 18981 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:46.654135 18981 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:46.656383 18981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "c308dd9289e34aa8b185aa9779ae9181"
format_stamp: "Formatted at 2026-08-11 22:13:46 on dist-test-slave-vxj2"
server_key: "cda1ce23c282f271e02f04271b005188"
server_key_iv: "5f03f7c7afaecebc169870216aa34405"
server_key_version: "encryptionkey@0"
I20260811 22:13:46.657008 18981 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "c308dd9289e34aa8b185aa9779ae9181"
format_stamp: "Formatted at 2026-08-11 22:13:46 on dist-test-slave-vxj2"
server_key: "cda1ce23c282f271e02f04271b005188"
server_key_iv: "5f03f7c7afaecebc169870216aa34405"
server_key_version: "encryptionkey@0"
I20260811 22:13:46.661506 18981 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:13:46.664410 18997 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:46.665973 18981 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:46.666136 18981 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "c308dd9289e34aa8b185aa9779ae9181"
format_stamp: "Formatted at 2026-08-11 22:13:46 on dist-test-slave-vxj2"
server_key: "cda1ce23c282f271e02f04271b005188"
server_key_iv: "5f03f7c7afaecebc169870216aa34405"
server_key_version: "encryptionkey@0"
I20260811 22:13:46.666296 18981 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:46.681691 18981 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:46.682472 18981 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:46.682694 18981 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:46.683362 18981 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:46.684413 18981 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:46.684501 18981 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:46.684602 18981 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:46.684654 18981 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:46.694666 18981 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:44437
I20260811 22:13:46.694692 19131 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:44437 every 8 connection(s)
I20260811 22:13:46.695750 18981 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:13:46.701246 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 18981
I20260811 22:13:46.701401 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:13:46.701711 28862 external_mini_cluster.cc:1428] Setting key e78be409e8a8d85bca052e0d312a7ba2
I20260811 22:13:46.704082 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42547
--builtin_ntp_servers=127.28.47.148:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:13:46.707232 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.697809 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:53297 (local address 127.28.47.190:42547)
0811 22:13:46.697966 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:46.697970 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:46.699120 (+ 1150us) server_negotiation.cc:407] Connection header received
0811 22:13:46.700621 (+ 1501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:46.700625 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:46.700715 (+ 90us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:46.700828 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:46.702007 (+ 1179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.702745 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.704655 (+ 1910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.704959 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.705058 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.706594 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:46.706600 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:46.706606 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:46.706627 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:46.706656 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:46.706670 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:46.706758 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:46.706926 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 22:13:46.707017 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:13:46.708348 19135 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.698209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42547 (local address 127.28.47.129:53297)
0811 22:13:46.698943 (+ 734us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:46.698989 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:46.700381 (+ 1392us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:46.701024 (+ 643us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:46.701032 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:46.701119 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:46.701841 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.701863 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.702882 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.702884 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.704482 (+ 1598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.704501 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.705147 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.705181 (+ 34us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.705435 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.706263 (+ 828us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:46.706265 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:46.706310 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:46.706387 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:46.706775 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:46.706783 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:46.706788 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:13:46.706889 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 22:13:46.707176 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":124,"threads_started":1}
I20260811 22:13:46.709830 19132 heartbeater.cc:347] Connected to a master server at 127.28.47.190:42547
I20260811 22:13:46.710104 19132 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:46.710707 19132 heartbeater.cc:511] Master 127.28.47.190:42547 requested a full tablet report, sending...
I20260811 22:13:46.712409 18909 ts_manager.cc:194] Registered new tserver with Master: c308dd9289e34aa8b185aa9779ae9181 (127.28.47.129:44437)
I20260811 22:13:46.713728 18909 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.47.129:53297
W20260811 22:13:46.824976 19136 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:46.825348 19136 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:46.825434 19136 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:46.829701 19136 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:46.829816 19136 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:46.829917 19136 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:13:46.834928 19136 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19136
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:46.836169 19136 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:46.837803 19136 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:46.847998 19141 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:46.847998 19147 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:46.848057 19143 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:46.848222 19136 server_base.cc:1034] running on GCE node
I20260811 22:13:46.848716 19136 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:46.849510 19136 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:46.850721 19136 hybrid_clock.cc:648] HybridClock initialized: now 1786486426850703 us; error 30 us; skew 500 ppm
I20260811 22:13:46.852887 19136 webserver.cc:460] Webserver started at http://127.28.47.130:41777/ using document root <none> and password file <none>
I20260811 22:13:46.853592 19136 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:46.853693 19136 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:46.854044 19136 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:46.856376 19136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "d29e9650b67243bebbce0afce0eca327"
format_stamp: "Formatted at 2026-08-11 22:13:46 on dist-test-slave-vxj2"
server_key: "b1729d2fd286c38e1a4ca4b2d82a898b"
server_key_iv: "d3a62d24b72f8ba2726d7d8f05a58b93"
server_key_version: "encryptionkey@0"
I20260811 22:13:46.857005 19136 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "d29e9650b67243bebbce0afce0eca327"
format_stamp: "Formatted at 2026-08-11 22:13:46 on dist-test-slave-vxj2"
server_key: "b1729d2fd286c38e1a4ca4b2d82a898b"
server_key_iv: "d3a62d24b72f8ba2726d7d8f05a58b93"
server_key_version: "encryptionkey@0"
I20260811 22:13:46.861047 19136 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260811 22:13:46.863888 19154 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:46.865424 19136 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:13:46.865572 19136 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "d29e9650b67243bebbce0afce0eca327"
format_stamp: "Formatted at 2026-08-11 22:13:46 on dist-test-slave-vxj2"
server_key: "b1729d2fd286c38e1a4ca4b2d82a898b"
server_key_iv: "d3a62d24b72f8ba2726d7d8f05a58b93"
server_key_version: "encryptionkey@0"
I20260811 22:13:46.865741 19136 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:46.900138 19136 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:46.900987 19136 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:46.901240 19136 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:46.901937 19136 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:46.902993 19136 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:46.903072 19136 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:46.903142 19136 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:46.903191 19136 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:46.912523 19136 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:44639
I20260811 22:13:46.912601 19291 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:44639 every 8 connection(s)
I20260811 22:13:46.913724 19136 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:13:46.923682 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 19136
I20260811 22:13:46.923830 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:13:46.923869 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.915805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:60189 (local address 127.28.47.190:42547)
0811 22:13:46.915967 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:46.915972 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:46.917219 (+ 1247us) server_negotiation.cc:407] Connection header received
0811 22:13:46.918711 (+ 1492us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:46.918714 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:46.918776 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:46.918866 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:46.919886 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.920440 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.921734 (+ 1294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.921971 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.922065 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.923246 (+ 1181us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:46.923249 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:46.923259 (+ 10us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:46.923273 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:46.923296 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:46.923304 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:46.923367 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:46.923612 (+ 245us) server_negotiation.cc:299] Negotiation successful
0811 22:13:46.923677 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:13:46.924149 28862 external_mini_cluster.cc:1428] Setting key 9b58b705f8ace9a430668e98f200a3a1
I20260811 22:13:46.924990 19294 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:46.916327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42547 (local address 127.28.47.130:60189)
0811 22:13:46.917023 (+ 696us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:46.917067 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:46.918474 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:46.919065 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:46.919073 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:46.919160 (+ 87us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:46.919681 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.919693 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.920567 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.920569 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.921604 (+ 1035us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:46.921615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:46.922124 (+ 509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:46.922127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:46.922269 (+ 142us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:46.922952 (+ 683us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:46.922954 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:46.923005 (+ 51us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:46.923074 (+ 69us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:46.923397 (+ 323us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:46.923407 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:46.923415 (+ 8us) client_negotiation.cc:769] Sending connection context
0811 22:13:46.923560 (+ 145us) client_negotiation.cc:240] Negotiation successful
0811 22:13:46.923936 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":122,"threads_started":1}
I20260811 22:13:46.926244 19292 heartbeater.cc:347] Connected to a master server at 127.28.47.190:42547
I20260811 22:13:46.926436 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42547
--builtin_ntp_servers=127.28.47.148:35237
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:13:46.926534 19292 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:46.927146 19292 heartbeater.cc:511] Master 127.28.47.190:42547 requested a full tablet report, sending...
I20260811 22:13:46.928355 18909 ts_manager.cc:194] Registered new tserver with Master: d29e9650b67243bebbce0afce0eca327 (127.28.47.130:44639)
I20260811 22:13:46.929214 18909 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.47.130:60189
W20260811 22:13:47.047899 19295 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:47.048183 19295 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:47.048233 19295 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:47.052384 19295 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:47.052456 19295 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:47.052515 19295 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:13:47.057637 19295 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:35237
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:42547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19295
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:47.058828 19295 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:47.060266 19295 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:47.074350 19300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:47.074363 19301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:47.074409 19303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:47.075372 19295 server_base.cc:1034] running on GCE node
I20260811 22:13:47.075924 19295 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:47.076825 19295 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:47.078040 19295 hybrid_clock.cc:648] HybridClock initialized: now 1786486427077954 us; error 100 us; skew 500 ppm
I20260811 22:13:47.080864 19295 webserver.cc:460] Webserver started at http://127.28.47.131:37441/ using document root <none> and password file <none>
I20260811 22:13:47.081735 19295 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:47.081841 19295 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:47.082125 19295 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:47.085639 19295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "dcd40b371ad546a8a2df0168190be5e9"
format_stamp: "Formatted at 2026-08-11 22:13:47 on dist-test-slave-vxj2"
server_key: "47679b082530d37db5f760febdfd574e"
server_key_iv: "2e8b2ba52a6f1a8f5ecbff9c3c3ab269"
server_key_version: "encryptionkey@0"
I20260811 22:13:47.086522 19295 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "dcd40b371ad546a8a2df0168190be5e9"
format_stamp: "Formatted at 2026-08-11 22:13:47 on dist-test-slave-vxj2"
server_key: "47679b082530d37db5f760febdfd574e"
server_key_iv: "2e8b2ba52a6f1a8f5ecbff9c3c3ab269"
server_key_version: "encryptionkey@0"
I20260811 22:13:47.092490 19295 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.006s sys 0.000s
I20260811 22:13:47.095990 19310 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:47.097800 19295 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260811 22:13:47.097966 19295 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "dcd40b371ad546a8a2df0168190be5e9"
format_stamp: "Formatted at 2026-08-11 22:13:47 on dist-test-slave-vxj2"
server_key: "47679b082530d37db5f760febdfd574e"
server_key_iv: "2e8b2ba52a6f1a8f5ecbff9c3c3ab269"
server_key_version: "encryptionkey@0"
I20260811 22:13:47.098140 19295 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:47.117786 19295 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:47.118561 19295 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:47.118788 19295 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:47.119445 19295 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:47.120476 19295 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:47.120527 19295 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:47.120599 19295 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:47.120656 19295 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:47.130335 19295 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:45027
I20260811 22:13:47.130357 19444 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:45027 every 8 connection(s)
I20260811 22:13:47.131325 19295 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:13:47.134999 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 19295
I20260811 22:13:47.135146 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:13:47.135474 28862 external_mini_cluster.cc:1428] Setting key 6d4db1220f1af9579fdd4ad497d77d64
I20260811 22:13:47.142481 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.133235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:37911 (local address 127.28.47.190:42547)
0811 22:13:47.133444 (+ 209us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:47.133448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:47.134729 (+ 1281us) server_negotiation.cc:407] Connection header received
0811 22:13:47.136666 (+ 1937us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:47.136669 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:47.136730 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:47.136814 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:47.138197 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.138901 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.140205 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.140452 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.140546 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.141864 (+ 1318us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:47.141867 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:47.141869 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:47.141886 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:47.141920 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:47.141929 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:47.142002 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:47.142273 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 22:13:47.142353 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112}
I20260811 22:13:47.143374 19450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.133655 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:42547 (local address 127.28.47.131:37911)
0811 22:13:47.134561 (+ 906us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:47.134617 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:47.136430 (+ 1813us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:47.137002 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:47.137008 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:47.137126 (+ 118us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:47.138038 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:47.138064 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.139036 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.139039 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:47.140062 (+ 1023us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:47.140072 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.140602 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.140605 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:47.140767 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.141552 (+ 785us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:47.141553 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:47.141600 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:47.141674 (+ 74us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:47.142024 (+ 350us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:47.142033 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:47.142038 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:13:47.142147 (+ 109us) client_negotiation.cc:240] Negotiation successful
0811 22:13:47.142420 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":338,"thread_start_us":162,"threads_started":1}
I20260811 22:13:47.144717 19445 heartbeater.cc:347] Connected to a master server at 127.28.47.190:42547
I20260811 22:13:47.144955 19445 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:47.145661 19445 heartbeater.cc:511] Master 127.28.47.190:42547 requested a full tablet report, sending...
I20260811 22:13:47.146868 18909 ts_manager.cc:194] Registered new tserver with Master: dcd40b371ad546a8a2df0168190be5e9 (127.28.47.131:45027)
I20260811 22:13:47.147511 18909 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.47.131:37911
I20260811 22:13:47.151424 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:13:47.157733 18957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.153587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35138 (local address 127.28.47.190:42547)
0811 22:13:47.153812 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:47.153819 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:47.153916 (+ 97us) server_negotiation.cc:407] Connection header received
0811 22:13:47.153996 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:47.153998 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:47.154043 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:47.154118 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:47.154386 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.154944 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.155831 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.156069 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.156153 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.157079 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:47.157082 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:47.157085 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:47.157098 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:47.157120 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:47.157128 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:47.157363 (+ 235us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:47.157560 (+ 197us) server_negotiation.cc:299] Negotiation successful
0811 22:13:47.157606 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
I20260811 22:13:47.161494 18909 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:35138:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:13:47.165346 18909 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:13:47.182085 19465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.176686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:45027 (local address 127.0.0.1:46980)
0811 22:13:47.177210 (+ 524us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:47.177223 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:47.177298 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:47.178037 (+ 739us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:47.178040 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:47.178056 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:47.178145 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:47.178151 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.179230 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.179234 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:47.180427 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:47.180438 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.181492 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.181496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:47.181618 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.181663 (+ 45us) client_negotiation.cc:769] Sending connection context
0811 22:13:47.181769 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:13:47.181909 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":98,"thread_start_us":108,"threads_started":1}
I20260811 22:13:47.182487 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.177124 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46980 (local address 127.28.47.131:45027)
0811 22:13:47.177585 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:47.177596 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:47.177618 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:13:47.177689 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:47.177695 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:47.177884 (+ 189us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:47.178065 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:47.178256 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.179107 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.180559 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.181343 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.181455 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.181661 (+ 206us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:47.182180 (+ 519us) server_negotiation.cc:299] Negotiation successful
0811 22:13:47.182295 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":346,"thread_start_us":185,"threads_started":1}
I20260811 22:13:47.182554 19468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.176347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:44639 (local address 127.0.0.1:42794)
0811 22:13:47.176964 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:47.176978 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:47.177088 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:47.177771 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:47.177773 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:47.177790 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:47.177873 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:47.177879 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.179233 (+ 1354us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.179236 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:47.180815 (+ 1579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:47.180825 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.181768 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.181772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:47.182225 (+ 453us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.182247 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:13:47.182321 (+ 74us) client_negotiation.cc:240] Negotiation successful
0811 22:13:47.182397 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":171,"spinlock_wait_cycles":15488,"thread_start_us":108,"threads_started":1}
I20260811 22:13:47.182740 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.175547 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:44437 (local address 127.0.0.1:47918)
0811 22:13:47.175947 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:47.176009 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:47.176154 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:47.177030 (+ 876us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:47.177039 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:47.177084 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:47.177554 (+ 470us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:47.177563 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.179024 (+ 1461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.179028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:47.180703 (+ 1675us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:47.180714 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.181768 (+ 1054us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.181772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:47.181889 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.181922 (+ 33us) client_negotiation.cc:769] Sending connection context
0811 22:13:47.181989 (+ 67us) client_negotiation.cc:240] Negotiation successful
0811 22:13:47.182065 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"mutex_wait_us":161,"spinlock_wait_cycles":3712,"thread_start_us":126,"threads_started":1}
I20260811 22:13:47.183015 19466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.176546 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42794 (local address 127.28.47.130:44639)
0811 22:13:47.177014 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:47.177027 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:47.177051 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:13:47.177268 (+ 217us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:47.177274 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:47.177469 (+ 195us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:47.177595 (+ 126us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:47.178039 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.179108 (+ 1069us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.180941 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.181651 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.181773 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.181970 (+ 197us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:47.182727 (+ 757us) server_negotiation.cc:299] Negotiation successful
0811 22:13:47.182837 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":145,"threads_started":1}
I20260811 22:13:47.183984 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.175752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47918 (local address 127.28.47.129:44437)
0811 22:13:47.176447 (+ 695us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:47.176457 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:47.176487 (+ 30us) server_negotiation.cc:407] Connection header received
0811 22:13:47.176571 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:47.176579 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:47.176826 (+ 247us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:47.177590 (+ 764us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:47.177752 (+ 162us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.178891 (+ 1139us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.180845 (+ 1954us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.181632 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.181841 (+ 209us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.182030 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:47.182080 (+ 50us) server_negotiation.cc:299] Negotiation successful
0811 22:13:47.182182 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":568,"thread_start_us":149,"threads_started":1}
I20260811 22:13:47.185829 19368 tablet_service.cc:1467] Processing CreateTablet for tablet 4039d4147e98467aafec9e7e9c2adb19 (DEFAULT_TABLE table=test-table [id=d37560388763461081c86cc5da0dfbc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:47.185827 19213 tablet_service.cc:1467] Processing CreateTablet for tablet 4039d4147e98467aafec9e7e9c2adb19 (DEFAULT_TABLE table=test-table [id=d37560388763461081c86cc5da0dfbc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:47.186345 19053 tablet_service.cc:1467] Processing CreateTablet for tablet 4039d4147e98467aafec9e7e9c2adb19 (DEFAULT_TABLE table=test-table [id=d37560388763461081c86cc5da0dfbc5]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:47.187065 19368 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4039d4147e98467aafec9e7e9c2adb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:47.187065 19213 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4039d4147e98467aafec9e7e9c2adb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:47.187420 19053 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4039d4147e98467aafec9e7e9c2adb19. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:47.194486 19472 tablet_bootstrap.cc:492] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181: Bootstrap starting.
I20260811 22:13:47.195416 19473 tablet_bootstrap.cc:492] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9: Bootstrap starting.
I20260811 22:13:47.196877 19472 tablet_bootstrap.cc:654] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:47.197346 19473 tablet_bootstrap.cc:654] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:47.197576 19474 tablet_bootstrap.cc:492] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327: Bootstrap starting.
I20260811 22:13:47.197942 19472 log.cc:824] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:47.198235 19473 log.cc:824] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:47.199892 19474 tablet_bootstrap.cc:654] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:47.200186 19472 tablet_bootstrap.cc:492] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181: No bootstrap required, opened a new log
I20260811 22:13:47.200320 19473 tablet_bootstrap.cc:492] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9: No bootstrap required, opened a new log
I20260811 22:13:47.200443 19472 ts_tablet_manager.cc:1397] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:13:47.200515 19473 ts_tablet_manager.cc:1397] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 22:13:47.200737 19474 log.cc:824] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:47.203073 19474 tablet_bootstrap.cc:492] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327: No bootstrap required, opened a new log
I20260811 22:13:47.203284 19474 ts_tablet_manager.cc:1397] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260811 22:13:47.204391 19472 raft_consensus.cc:348] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c308dd9289e34aa8b185aa9779ae9181" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 44437 } } peers { permanent_uuid: "d29e9650b67243bebbce0afce0eca327" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 44639 } } peers { permanent_uuid: "dcd40b371ad546a8a2df0168190be5e9" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45027 } }
I20260811 22:13:47.204653 19472 raft_consensus.cc:374] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:47.204708 19472 raft_consensus.cc:729] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c308dd9289e34aa8b185aa9779ae9181, State: Initialized, Role: FOLLOWER
I20260811 22:13:47.205039 19473 raft_consensus.cc:348] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c308dd9289e34aa8b185aa9779ae9181" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 44437 } } peers { permanent_uuid: "d29e9650b67243bebbce0afce0eca327" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 44639 } } peers { permanent_uuid: "dcd40b371ad546a8a2df0168190be5e9" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45027 } }
I20260811 22:13:47.205318 19473 raft_consensus.cc:374] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:47.205261 19472 consensus_queue.cc:260] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c308dd9289e34aa8b185aa9779ae9181" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 44437 } } peers { permanent_uuid: "d29e9650b67243bebbce0afce0eca327" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 44639 } } peers { permanent_uuid: "dcd40b371ad546a8a2df0168190be5e9" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45027 } }
I20260811 22:13:47.205379 19473 raft_consensus.cc:729] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dcd40b371ad546a8a2df0168190be5e9, State: Initialized, Role: FOLLOWER
I20260811 22:13:47.205839 19473 consensus_queue.cc:260] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c308dd9289e34aa8b185aa9779ae9181" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 44437 } } peers { permanent_uuid: "d29e9650b67243bebbce0afce0eca327" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 44639 } } peers { permanent_uuid: "dcd40b371ad546a8a2df0168190be5e9" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45027 } }
I20260811 22:13:47.206403 19472 ts_tablet_manager.cc:1428] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260811 22:13:47.206799 19473 ts_tablet_manager.cc:1428] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260811 22:13:47.206808 19132 heartbeater.cc:503] Master 127.28.47.190:42547 was elected leader, sending a full tablet report...
I20260811 22:13:47.207248 19445 heartbeater.cc:503] Master 127.28.47.190:42547 was elected leader, sending a full tablet report...
I20260811 22:13:47.207715 19474 raft_consensus.cc:348] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c308dd9289e34aa8b185aa9779ae9181" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 44437 } } peers { permanent_uuid: "d29e9650b67243bebbce0afce0eca327" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 44639 } } peers { permanent_uuid: "dcd40b371ad546a8a2df0168190be5e9" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45027 } }
I20260811 22:13:47.207968 19474 raft_consensus.cc:374] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:47.208034 19474 raft_consensus.cc:729] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d29e9650b67243bebbce0afce0eca327, State: Initialized, Role: FOLLOWER
I20260811 22:13:47.208530 19474 consensus_queue.cc:260] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c308dd9289e34aa8b185aa9779ae9181" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 44437 } } peers { permanent_uuid: "d29e9650b67243bebbce0afce0eca327" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 44639 } } peers { permanent_uuid: "dcd40b371ad546a8a2df0168190be5e9" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45027 } }
I20260811 22:13:47.209362 19292 heartbeater.cc:503] Master 127.28.47.190:42547 was elected leader, sending a full tablet report...
I20260811 22:13:47.209604 19474 ts_tablet_manager.cc:1428] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260811 22:13:47.371387 19481 raft_consensus.cc:482] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:13:47.371637 19481 raft_consensus.cc:504] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c308dd9289e34aa8b185aa9779ae9181" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 44437 } } peers { permanent_uuid: "d29e9650b67243bebbce0afce0eca327" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 44639 } } peers { permanent_uuid: "dcd40b371ad546a8a2df0168190be5e9" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45027 } }
I20260811 22:13:47.372989 19481 leader_election.cc:288] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c308dd9289e34aa8b185aa9779ae9181 (127.28.47.129:44437), d29e9650b67243bebbce0afce0eca327 (127.28.47.130:44639)
I20260811 22:13:47.376892 19450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.373253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:44437 (local address 127.28.47.131:37717)
0811 22:13:47.373423 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:47.373444 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:47.373553 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:47.374053 (+ 500us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:47.374056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:47.374077 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:47.374208 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:47.374214 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.374984 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.374990 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:47.375778 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:47.375788 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.376448 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.376452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:47.376554 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.376584 (+ 30us) client_negotiation.cc:769] Sending connection context
0811 22:13:47.376674 (+ 90us) client_negotiation.cc:240] Negotiation successful
0811 22:13:47.376751 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 22:13:47.376891 19464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.373353 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:37717 (local address 127.28.47.129:44437)
0811 22:13:47.373523 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:47.373528 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:47.373544 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:13:47.373883 (+ 339us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:47.373888 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:47.373950 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:47.374038 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:47.374336 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.374853 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.375913 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.376316 (+ 403us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.376455 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.376512 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:47.376692 (+ 180us) server_negotiation.cc:299] Negotiation successful
0811 22:13:47.376764 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:13:47.377501 19466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.373741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:43853 (local address 127.28.47.130:44639)
0811 22:13:47.373872 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:47.373876 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:47.374136 (+ 260us) server_negotiation.cc:407] Connection header received
0811 22:13:47.374309 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:47.374312 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:47.374373 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:47.374466 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:47.374797 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.375312 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.376387 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.376803 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.376958 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.377014 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:47.377262 (+ 248us) server_negotiation.cc:299] Negotiation successful
0811 22:13:47.377349 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:13:47.377700 19078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4039d4147e98467aafec9e7e9c2adb19" candidate_uuid: "dcd40b371ad546a8a2df0168190be5e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c308dd9289e34aa8b185aa9779ae9181" is_pre_election: true
I20260811 22:13:47.377935 19483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.373614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:44639 (local address 127.28.47.131:43853)
0811 22:13:47.374022 (+ 408us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:47.374035 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:47.374177 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:47.374479 (+ 302us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:47.374482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:47.374500 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:47.374657 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:47.374663 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.375441 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.375447 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:47.376243 (+ 796us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:47.376253 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.376958 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.376963 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:47.377067 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.377096 (+ 29us) client_negotiation.cc:769] Sending connection context
0811 22:13:47.377689 (+ 593us) client_negotiation.cc:240] Negotiation successful
0811 22:13:47.377776 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":180,"threads_started":1}
I20260811 22:13:47.378126 19078 raft_consensus.cc:2454] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcd40b371ad546a8a2df0168190be5e9 in term 0.
I20260811 22:13:47.378706 19313 leader_election.cc:302] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c308dd9289e34aa8b185aa9779ae9181, dcd40b371ad546a8a2df0168190be5e9; no voters:
I20260811 22:13:47.378681 19240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4039d4147e98467aafec9e7e9c2adb19" candidate_uuid: "dcd40b371ad546a8a2df0168190be5e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d29e9650b67243bebbce0afce0eca327" is_pre_election: true
I20260811 22:13:47.379010 19240 raft_consensus.cc:2454] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dcd40b371ad546a8a2df0168190be5e9 in term 0.
I20260811 22:13:47.379037 19481 raft_consensus.cc:2781] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:13:47.379141 19481 raft_consensus.cc:482] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:13:47.379206 19481 raft_consensus.cc:3037] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:47.380429 19481 raft_consensus.cc:504] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c308dd9289e34aa8b185aa9779ae9181" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 44437 } } peers { permanent_uuid: "d29e9650b67243bebbce0afce0eca327" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 44639 } } peers { permanent_uuid: "dcd40b371ad546a8a2df0168190be5e9" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45027 } }
I20260811 22:13:47.380798 19481 leader_election.cc:288] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [CANDIDATE]: Term 1 election: Requested vote from peers c308dd9289e34aa8b185aa9779ae9181 (127.28.47.129:44437), d29e9650b67243bebbce0afce0eca327 (127.28.47.130:44639)
I20260811 22:13:47.381424 19240 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4039d4147e98467aafec9e7e9c2adb19" candidate_uuid: "dcd40b371ad546a8a2df0168190be5e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d29e9650b67243bebbce0afce0eca327"
I20260811 22:13:47.381423 19078 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4039d4147e98467aafec9e7e9c2adb19" candidate_uuid: "dcd40b371ad546a8a2df0168190be5e9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c308dd9289e34aa8b185aa9779ae9181"
I20260811 22:13:47.381585 19078 raft_consensus.cc:3037] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:47.381587 19240 raft_consensus.cc:3037] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:47.383294 19240 raft_consensus.cc:2454] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcd40b371ad546a8a2df0168190be5e9 in term 1.
I20260811 22:13:47.383308 19078 raft_consensus.cc:2454] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dcd40b371ad546a8a2df0168190be5e9 in term 1.
I20260811 22:13:47.383723 19313 leader_election.cc:302] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: c308dd9289e34aa8b185aa9779ae9181, dcd40b371ad546a8a2df0168190be5e9; no voters:
I20260811 22:13:47.383957 19481 raft_consensus.cc:2781] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:47.384313 19481 raft_consensus.cc:686] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [term 1 LEADER]: Becoming Leader. State: Replica: dcd40b371ad546a8a2df0168190be5e9, State: Running, Role: LEADER
I20260811 22:13:47.384778 19481 consensus_queue.cc:237] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c308dd9289e34aa8b185aa9779ae9181" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 44437 } } peers { permanent_uuid: "d29e9650b67243bebbce0afce0eca327" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 44639 } } peers { permanent_uuid: "dcd40b371ad546a8a2df0168190be5e9" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45027 } }
W20260811 22:13:47.385221 19448 tablet.cc:2369] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:13:47.389015 18909 catalog_manager.cc:5495] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 reported cstate change: term changed from 0 to 1, leader changed from <none> to dcd40b371ad546a8a2df0168190be5e9 (127.28.47.131). New cstate: current_term: 1 leader_uuid: "dcd40b371ad546a8a2df0168190be5e9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c308dd9289e34aa8b185aa9779ae9181" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 44437 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d29e9650b67243bebbce0afce0eca327" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 44639 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dcd40b371ad546a8a2df0168190be5e9" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 45027 } health_report { overall_health: HEALTHY } } }
W20260811 22:13:47.415266 19293 tablet.cc:2369] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:13:47.422201 19470 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.418577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46982 (local address 127.28.47.131:45027)
0811 22:13:47.418717 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:47.418720 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:47.418735 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:13:47.418808 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:47.418810 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:47.418872 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:47.418969 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:13:47.419221 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.419755 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.420650 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:47.420873 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:47.420967 (+ 94us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:47.421375 (+ 408us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:13:47.421826 (+ 451us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:13:47.421958 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:47.422051 (+ 93us) server_negotiation.cc:299] Negotiation successful
0811 22:13:47.422085 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260811 22:13:47.428594 19240 raft_consensus.cc:1261] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327 [term 1 FOLLOWER]: Refusing update from remote peer dcd40b371ad546a8a2df0168190be5e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:47.428625 19078 raft_consensus.cc:1261] T 4039d4147e98467aafec9e7e9c2adb19 P c308dd9289e34aa8b185aa9779ae9181 [term 1 FOLLOWER]: Refusing update from remote peer dcd40b371ad546a8a2df0168190be5e9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:47.429384 19481 consensus_queue.cc:1035] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d29e9650b67243bebbce0afce0eca327" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 44639 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:47.429592 19484 consensus_queue.cc:1035] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c308dd9289e34aa8b185aa9779ae9181" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 44437 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:47.448347 18907 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:35138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:47.448555 18907 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:35138:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:47.451468 18907 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 0ed8a73a91304c38af4b30bcad8f1083: Sending DeleteTablet for 3 replicas of tablet 4039d4147e98467aafec9e7e9c2adb19
I20260811 22:13:47.452870 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 18981
I20260811 22:13:47.453538 19213 tablet_service.cc:1514] Processing DeleteTablet for tablet 4039d4147e98467aafec9e7e9c2adb19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:47 UTC) from {username='slave'} at 127.0.0.1:42794
I20260811 22:13:47.454073 19368 tablet_service.cc:1514] Processing DeleteTablet for tablet 4039d4147e98467aafec9e7e9c2adb19 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:47 UTC) from {username='slave'} at 127.0.0.1:46980
I20260811 22:13:47.457365 19498 tablet_replica.cc:330] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327: stopping tablet replica
I20260811 22:13:47.457777 19498 raft_consensus.cc:2229] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:47.458189 19498 raft_consensus.cc:2258] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:47.459964 19498 ts_tablet_manager.cc:1905] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:47.463363 19499 tablet_replica.cc:330] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9: stopping tablet replica
I20260811 22:13:47.463766 19499 raft_consensus.cc:2229] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [term 1 LEADER]: Raft consensus shutting down.
I20260811 22:13:47.464294 19498 ts_tablet_manager.cc:1918] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:47.464339 19499 raft_consensus.cc:2258] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:47.464545 19498 log.cc:1196] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/4039d4147e98467aafec9e7e9c2adb19
I20260811 22:13:47.464985 19498 ts_tablet_manager.cc:1939] T 4039d4147e98467aafec9e7e9c2adb19 P d29e9650b67243bebbce0afce0eca327: Deleting consensus metadata
I20260811 22:13:47.466229 19499 ts_tablet_manager.cc:1905] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:47.466821 18893 catalog_manager.cc:4841] TS d29e9650b67243bebbce0afce0eca327 (127.28.47.130:44639): tablet 4039d4147e98467aafec9e7e9c2adb19 (table test-table [id=d37560388763461081c86cc5da0dfbc5]) successfully deleted
W20260811 22:13:47.467142 18893 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:44437 (error 108)
I20260811 22:13:47.467438 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 19136
I20260811 22:13:47.472447 19463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:47.471974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:44437 (local address 127.0.0.1:47926)
0811 22:13:47.472243 (+ 269us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:47.472327 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:44437: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145}
W20260811 22:13:47.472671 18893 catalog_manager.cc:4584] TS c308dd9289e34aa8b185aa9779ae9181 (127.28.47.129:44437): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4039d4147e98467aafec9e7e9c2adb19: Network error: Client connection negotiation failed: client connection to 127.28.47.129:44437: connect: Connection refused (error 111)
I20260811 22:13:47.479775 19499 ts_tablet_manager.cc:1918] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:47.479959 19499 log.cc:1196] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/4039d4147e98467aafec9e7e9c2adb19
I20260811 22:13:47.480300 19499 ts_tablet_manager.cc:1939] T 4039d4147e98467aafec9e7e9c2adb19 P dcd40b371ad546a8a2df0168190be5e9: Deleting consensus metadata
I20260811 22:13:47.481829 18893 catalog_manager.cc:4841] TS dcd40b371ad546a8a2df0168190be5e9 (127.28.47.131:45027): tablet 4039d4147e98467aafec9e7e9c2adb19 (table test-table [id=d37560388763461081c86cc5da0dfbc5]) successfully deleted
I20260811 22:13:47.484910 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 19295
I20260811 22:13:47.494642 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 18876
2026-08-11T22:13:47Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1279 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:13:47 dist-test-slave-vxj2 krb5kdc[19507](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:13:47 dist-test-slave-vxj2 krb5kdc[19507](info): set up 2 sockets
Aug 11 22:13:47 dist-test-slave-vxj2 krb5kdc[19507](info): commencing operation
krb5kdc: starting...
W20260811 22:13:48.642843 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.123s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:13:48 dist-test-slave-vxj2 krb5kdc[19507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:13:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:13:48.785259 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:33061
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:43975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:33061
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:48.905642 19524 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:48.906014 19524 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:48.906091 19524 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:48.910367 19524 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:48.910492 19524 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:48.910529 19524 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:48.910569 19524 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:48.910629 19524 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:48.916527 19524 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:43975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:33061
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:33061
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19524
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:48.917793 19524 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:48.918979 19524 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:48.928366 19530 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:48.928462 19532 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:48.928452 19524 server_base.cc:1034] running on GCE node
W20260811 22:13:48.928396 19529 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:48.929522 19524 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:48.930733 19524 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:48.931922 19524 hybrid_clock.cc:648] HybridClock initialized: now 1786486428931903 us; error 36 us; skew 500 ppm
Aug 11 22:13:48 dist-test-slave-vxj2 krb5kdc[19507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486428, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:48.940299 19524 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:13:48.941439 19524 webserver.cc:460] Webserver started at http://127.28.47.190:46013/ using document root <none> and password file <none>
I20260811 22:13:48.942072 19524 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:48.942148 19524 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:48.942353 19524 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:48.944734 19524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "10862fd4e8f8449887e228622a055e08"
format_stamp: "Formatted at 2026-08-11 22:13:48 on dist-test-slave-vxj2"
server_key: "76101c7b0ae75e30611b07c8f7b2bca5"
server_key_iv: "4b7b6e4f9b1344ff49f04b1fec4e323b"
server_key_version: "encryptionkey@0"
I20260811 22:13:48.945436 19524 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "10862fd4e8f8449887e228622a055e08"
format_stamp: "Formatted at 2026-08-11 22:13:48 on dist-test-slave-vxj2"
server_key: "76101c7b0ae75e30611b07c8f7b2bca5"
server_key_iv: "4b7b6e4f9b1344ff49f04b1fec4e323b"
server_key_version: "encryptionkey@0"
I20260811 22:13:48.949903 19524 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260811 22:13:48.952850 19541 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:48.954375 19524 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260811 22:13:48.954551 19524 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "10862fd4e8f8449887e228622a055e08"
format_stamp: "Formatted at 2026-08-11 22:13:48 on dist-test-slave-vxj2"
server_key: "76101c7b0ae75e30611b07c8f7b2bca5"
server_key_iv: "4b7b6e4f9b1344ff49f04b1fec4e323b"
server_key_version: "encryptionkey@0"
I20260811 22:13:48.954697 19524 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:48.985952 19524 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:48.989825 19524 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:48.990097 19524 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:48.999145 19524 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:33061
I20260811 22:13:48.999202 19610 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:33061 every 8 connection(s)
I20260811 22:13:49.000345 19524 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:49.003530 19611 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:49.003767 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 19524
I20260811 22:13:49.003902 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:13:49.004235 28862 external_mini_cluster.cc:1428] Setting key 5c3a365120cd741a4b312de2dd98968f
I20260811 22:13:49.010677 19611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08: Bootstrap starting.
I20260811 22:13:49.013341 19611 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08: Neither blocks nor log segments found. Creating new log.
Aug 11 22:13:49 dist-test-slave-vxj2 krb5kdc[19507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486428, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:13:49.014292 19611 log.cc:824] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:49.016512 19611 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08: No bootstrap required, opened a new log
I20260811 22:13:49.019379 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.005845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50752 (local address 127.28.47.190:33061)
0811 22:13:49.006360 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:49.006383 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:49.006420 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:13:49.007192 (+ 772us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:49.007216 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:49.007558 (+ 342us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:49.007913 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:49.008795 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.009868 (+ 1073us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.010780 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.011309 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.011483 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.014441 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:49.014449 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:49.014467 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:49.014504 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:49.016396 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:49.016938 (+ 542us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:49.016943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:49.016948 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:49.017069 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:49.017399 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:49.017403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:49.017405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:49.017837 (+ 432us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:49.018053 (+ 216us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:49.018340 (+ 287us) server_negotiation.cc:299] Negotiation successful
0811 22:13:49.018655 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":360,"thread_start_us":158,"threads_started":1}
I20260811 22:13:49.020581 19611 raft_consensus.cc:348] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10862fd4e8f8449887e228622a055e08" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 33061 } }
I20260811 22:13:49.020848 19611 raft_consensus.cc:374] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:49.020936 19611 raft_consensus.cc:729] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 10862fd4e8f8449887e228622a055e08, State: Initialized, Role: FOLLOWER
I20260811 22:13:49.021437 19611 consensus_queue.cc:260] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10862fd4e8f8449887e228622a055e08" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 33061 } }
I20260811 22:13:49.021591 19611 raft_consensus.cc:388] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:49.021683 19611 raft_consensus.cc:482] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:49.021799 19611 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:49.022821 19611 raft_consensus.cc:504] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10862fd4e8f8449887e228622a055e08" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 33061 } }
I20260811 22:13:49.023139 19611 leader_election.cc:302] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 10862fd4e8f8449887e228622a055e08; no voters:
I20260811 22:13:49.023408 19611 leader_election.cc:288] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:49.023533 19617 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:49.023823 19617 raft_consensus.cc:686] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [term 1 LEADER]: Becoming Leader. State: Replica: 10862fd4e8f8449887e228622a055e08, State: Running, Role: LEADER
I20260811 22:13:49.024140 19617 consensus_queue.cc:237] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10862fd4e8f8449887e228622a055e08" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 33061 } }
I20260811 22:13:49.024763 19611 sys_catalog.cc:564] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:49.026062 19619 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 10862fd4e8f8449887e228622a055e08. Latest consensus state: current_term: 1 leader_uuid: "10862fd4e8f8449887e228622a055e08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10862fd4e8f8449887e228622a055e08" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 33061 } } }
I20260811 22:13:49.026062 19618 sys_catalog.cc:455] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "10862fd4e8f8449887e228622a055e08" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "10862fd4e8f8449887e228622a055e08" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 33061 } } }
I20260811 22:13:49.026233 19619 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:49.026242 19618 sys_catalog.cc:458] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:49.026882 19626 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:49.030282 19626 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:49.036597 19626 catalog_manager.cc:1347] Generated new cluster ID: d721a50906ce457da5b6b11ef0d546eb
I20260811 22:13:49.036691 19626 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:49.072304 19626 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:49.073540 19626 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:49.087096 19626 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08: Generated new TSK 0
I20260811 22:13:49.088019 19626 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.28.47.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
Entry for principal kudu/127.28.47.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab.
I20260811 22:13:49.128739 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:33061
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:43975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:49.253826 19639 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:49.254132 19639 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:49.254180 19639 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:49.258661 19639 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:49.258729 19639 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:49.258790 19639 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:13:49.264390 19639 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:43975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.129.keytab
--principal=kudu/127.28.47.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:33061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19639
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:49.265588 19639 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:49.266767 19639 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:49.277277 19648 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:49.277410 19639 server_base.cc:1034] running on GCE node
W20260811 22:13:49.277215 19646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:49.277215 19651 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:49.278023 19639 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:49.278749 19639 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:49.279958 19639 hybrid_clock.cc:648] HybridClock initialized: now 1786486429279938 us; error 30 us; skew 500 ppm
Aug 11 22:13:49 dist-test-slave-vxj2 krb5kdc[19507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486429, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:49.283022 19639 init.cc:377] Logged in from keytab as kudu/127.28.47.129@KRBTEST.COM (short username kudu)
I20260811 22:13:49.284065 19639 webserver.cc:460] Webserver started at http://127.28.47.129:38055/ using document root <none> and password file <none>
I20260811 22:13:49.284683 19639 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:49.284780 19639 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:49.285036 19639 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:49.287465 19639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "0370ed960dca45abac71c5f40fc46cb0"
format_stamp: "Formatted at 2026-08-11 22:13:49 on dist-test-slave-vxj2"
server_key: "fd25891fefc0263b539612d716191c93"
server_key_iv: "43b38b94642b021ebe13e34d0aa896e1"
server_key_version: "encryptionkey@0"
I20260811 22:13:49.288102 19639 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "0370ed960dca45abac71c5f40fc46cb0"
format_stamp: "Formatted at 2026-08-11 22:13:49 on dist-test-slave-vxj2"
server_key: "fd25891fefc0263b539612d716191c93"
server_key_iv: "43b38b94642b021ebe13e34d0aa896e1"
server_key_version: "encryptionkey@0"
I20260811 22:13:49.292289 19639 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:13:49.294993 19658 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:49.296327 19639 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:13:49.296512 19639 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "0370ed960dca45abac71c5f40fc46cb0"
format_stamp: "Formatted at 2026-08-11 22:13:49 on dist-test-slave-vxj2"
server_key: "fd25891fefc0263b539612d716191c93"
server_key_iv: "43b38b94642b021ebe13e34d0aa896e1"
server_key_version: "encryptionkey@0"
I20260811 22:13:49.296659 19639 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:49.315240 19639 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:49.319074 19639 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:49.319327 19639 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:49.320029 19639 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:49.320974 19639 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:49.321053 19639 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:49.321130 19639 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:49.321216 19639 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.001s
I20260811 22:13:49.331488 19639 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:34073
I20260811 22:13:49.331508 19788 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:34073 every 8 connection(s)
I20260811 22:13:49.332512 19639 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:13:49.337592 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 19639
I20260811 22:13:49.337759 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:13:49.338101 28862 external_mini_cluster.cc:1428] Setting key d70fa335c5ea0c1179bc38fd3c3336b9
Aug 11 22:13:49 dist-test-slave-vxj2 krb5kdc[19507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486429, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.129@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:13:49.349327 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.334703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:47193 (local address 127.28.47.190:33061)
0811 22:13:49.334876 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:49.334881 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:49.335896 (+ 1015us) server_negotiation.cc:407] Connection header received
0811 22:13:49.337418 (+ 1522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:49.337421 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:49.337477 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:49.337567 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:49.339287 (+ 1720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.339852 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.341110 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.341435 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.341586 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.344583 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:49.344590 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:49.344596 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:49.344636 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:49.346632 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:49.347258 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:49.347263 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:49.347266 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:49.347370 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:49.347800 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:49.347803 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:49.347805 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:49.348126 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:49.348251 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:49.348850 (+ 599us) server_negotiation.cc:299] Negotiation successful
0811 22:13:49.349040 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
WARNING: no policy specified for kudu/127.28.47.130@KRBTEST.COM; defaulting to no policy
I20260811 22:13:49.350198 19791 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.335189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:33061 (local address 127.28.47.129:47193)
0811 22:13:49.335714 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:49.335764 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:49.337077 (+ 1313us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:49.337833 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:49.337844 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:49.338401 (+ 557us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:49.339122 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.339140 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.340019 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.340021 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.340967 (+ 946us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.340979 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.341670 (+ 691us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.341677 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.341898 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.342578 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:49.342596 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:49.344356 (+ 1760us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:49.346794 (+ 2438us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:49.346801 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:49.346816 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:49.347135 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:49.347510 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:49.347513 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:49.347515 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:49.347683 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:49.348256 (+ 573us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:49.348263 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:49.348567 (+ 304us) client_negotiation.cc:769] Sending connection context
0811 22:13:49.348745 (+ 178us) client_negotiation.cc:240] Negotiation successful
0811 22:13:49.349066 (+ 321us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":314,"thread_start_us":125,"threads_started":1}
I20260811 22:13:49.351729 19789 heartbeater.cc:347] Connected to a master server at 127.28.47.190:33061
I20260811 22:13:49.352035 19789 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:49.352641 19789 heartbeater.cc:511] Master 127.28.47.190:33061 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.130@KRBTEST.COM" created.
I20260811 22:13:49.354771 19564 ts_manager.cc:194] Registered new tserver with Master: 0370ed960dca45abac71c5f40fc46cb0 (127.28.47.129:34073)
I20260811 22:13:49.356249 19564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.129@KRBTEST.COM'} at 127.28.47.129:47193
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
Entry for principal kudu/127.28.47.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab.
I20260811 22:13:49.373943 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:33061
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:43975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:49.496769 19794 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:49.497159 19794 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:49.497269 19794 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:49.501780 19794 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:49.501896 19794 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:49.501999 19794 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:13:49.507948 19794 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:43975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.130.keytab
--principal=kudu/127.28.47.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:33061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19794
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:49.509297 19794 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:49.510442 19794 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:49.521562 19804 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:49.521562 19806 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:49.521564 19803 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:49.521857 19794 server_base.cc:1034] running on GCE node
I20260811 22:13:49.522387 19794 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:49.523126 19794 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:49.524361 19794 hybrid_clock.cc:648] HybridClock initialized: now 1786486429524307 us; error 33 us; skew 500 ppm
Aug 11 22:13:49 dist-test-slave-vxj2 krb5kdc[19507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486429, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:49.527695 19794 init.cc:377] Logged in from keytab as kudu/127.28.47.130@KRBTEST.COM (short username kudu)
I20260811 22:13:49.528683 19794 webserver.cc:460] Webserver started at http://127.28.47.130:34283/ using document root <none> and password file <none>
I20260811 22:13:49.529346 19794 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:49.529419 19794 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:49.529608 19794 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:49.531780 19794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "e63e583098b1429ea959258ae35c674b"
format_stamp: "Formatted at 2026-08-11 22:13:49 on dist-test-slave-vxj2"
server_key: "8fc885db53fc8b9ada93ef9a928320a8"
server_key_iv: "6cf720b938262499460b2c8196ae91da"
server_key_version: "encryptionkey@0"
I20260811 22:13:49.532351 19794 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "e63e583098b1429ea959258ae35c674b"
format_stamp: "Formatted at 2026-08-11 22:13:49 on dist-test-slave-vxj2"
server_key: "8fc885db53fc8b9ada93ef9a928320a8"
server_key_iv: "6cf720b938262499460b2c8196ae91da"
server_key_version: "encryptionkey@0"
I20260811 22:13:49.536418 19794 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:13:49.539237 19816 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:49.540802 19794 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:13:49.540977 19794 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "e63e583098b1429ea959258ae35c674b"
format_stamp: "Formatted at 2026-08-11 22:13:49 on dist-test-slave-vxj2"
server_key: "8fc885db53fc8b9ada93ef9a928320a8"
server_key_iv: "6cf720b938262499460b2c8196ae91da"
server_key_version: "encryptionkey@0"
I20260811 22:13:49.541110 19794 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:49.558924 19794 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:49.563081 19794 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:49.563333 19794 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:49.564062 19794 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:49.565045 19794 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:49.565099 19794 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:49.565200 19794 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:49.565243 19794 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:49.576128 19794 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:36597
I20260811 22:13:49.576148 19945 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:36597 every 8 connection(s)
I20260811 22:13:49.577299 19794 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:13:49.582464 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 19794
I20260811 22:13:49.582626 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:13:49.582966 28862 external_mini_cluster.cc:1428] Setting key a5e2aff179d6a1b0f0b9c5b0b8a90a82
Aug 11 22:13:49 dist-test-slave-vxj2 krb5kdc[19507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486429, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.130@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:13:49.593355 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.579479 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:44205 (local address 127.28.47.190:33061)
0811 22:13:49.579663 (+ 184us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:49.579668 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:49.580646 (+ 978us) server_negotiation.cc:407] Connection header received
0811 22:13:49.582083 (+ 1437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:49.582086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:49.582139 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:49.582219 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:49.583794 (+ 1575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.584428 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.585739 (+ 1311us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.586175 (+ 436us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.586312 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.589487 (+ 3175us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:49.589491 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:49.589493 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:49.589518 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:49.591029 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:49.591619 (+ 590us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:49.591623 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:49.591625 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:49.591709 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:49.592087 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:49.592090 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:49.592092 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:49.592305 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:49.592418 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:49.593004 (+ 586us) server_negotiation.cc:299] Negotiation successful
0811 22:13:49.593145 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
WARNING: no policy specified for kudu/127.28.47.131@KRBTEST.COM; defaulting to no policy
I20260811 22:13:49.594278 19948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.579904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:33061 (local address 127.28.47.130:44205)
0811 22:13:49.580455 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:49.580505 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:49.581849 (+ 1344us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:49.582474 (+ 625us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:49.582485 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:49.583027 (+ 542us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:49.583652 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.583668 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.584570 (+ 902us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.584574 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.585592 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.585604 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.586367 (+ 763us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.586370 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.586602 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.587286 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:49.587314 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:49.589291 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:49.591166 (+ 1875us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:49.591173 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:49.591192 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:49.591492 (+ 300us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:49.591810 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:49.591814 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:49.591816 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:49.591978 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:49.592419 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:49.592427 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:49.592727 (+ 300us) client_negotiation.cc:769] Sending connection context
0811 22:13:49.592942 (+ 215us) client_negotiation.cc:240] Negotiation successful
0811 22:13:49.593363 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":129,"threads_started":1}
I20260811 22:13:49.595598 19946 heartbeater.cc:347] Connected to a master server at 127.28.47.190:33061
I20260811 22:13:49.595906 19946 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:49.596488 19946 heartbeater.cc:511] Master 127.28.47.190:33061 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.131@KRBTEST.COM" created.
I20260811 22:13:49.597970 19564 ts_manager.cc:194] Registered new tserver with Master: e63e583098b1429ea959258ae35c674b (127.28.47.130:36597)
I20260811 22:13:49.598661 19564 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.130@KRBTEST.COM'} at 127.28.47.130:44205
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
Entry for principal kudu/127.28.47.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab.
I20260811 22:13:49.617587 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:33061
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:43975
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:49.737718 19951 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:49.738029 19951 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:49.738072 19951 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:49.742161 19951 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:49.742231 19951 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:49.742295 19951 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:13:49.748214 19951 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:43975
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.131.keytab
--principal=kudu/127.28.47.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:33061
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19951
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:49.749374 19951 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:49.750389 19951 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:49.761323 19960 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:49.761368 19957 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:49.761384 19951 server_base.cc:1034] running on GCE node
W20260811 22:13:49.761329 19956 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:49.762116 19951 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:49.762885 19951 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:49.764081 19951 hybrid_clock.cc:648] HybridClock initialized: now 1786486429764032 us; error 65 us; skew 500 ppm
Aug 11 22:13:49 dist-test-slave-vxj2 krb5kdc[19507](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486429, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:49.767285 19951 init.cc:377] Logged in from keytab as kudu/127.28.47.131@KRBTEST.COM (short username kudu)
I20260811 22:13:49.768373 19951 webserver.cc:460] Webserver started at http://127.28.47.131:41997/ using document root <none> and password file <none>
I20260811 22:13:49.769032 19951 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:49.769134 19951 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:49.769384 19951 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:49.771541 19951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "85e229dd08af426f9846f78f39196449"
format_stamp: "Formatted at 2026-08-11 22:13:49 on dist-test-slave-vxj2"
server_key: "54ed86420fec967ed8b889604a1c1526"
server_key_iv: "99b0f6d9f6f81940f6e958f5475ff002"
server_key_version: "encryptionkey@0"
I20260811 22:13:49.772138 19951 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "85e229dd08af426f9846f78f39196449"
format_stamp: "Formatted at 2026-08-11 22:13:49 on dist-test-slave-vxj2"
server_key: "54ed86420fec967ed8b889604a1c1526"
server_key_iv: "99b0f6d9f6f81940f6e958f5475ff002"
server_key_version: "encryptionkey@0"
I20260811 22:13:49.776176 19951 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.002s
I20260811 22:13:49.778873 19967 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:49.780319 19951 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:13:49.780505 19951 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "85e229dd08af426f9846f78f39196449"
format_stamp: "Formatted at 2026-08-11 22:13:49 on dist-test-slave-vxj2"
server_key: "54ed86420fec967ed8b889604a1c1526"
server_key_iv: "99b0f6d9f6f81940f6e958f5475ff002"
server_key_version: "encryptionkey@0"
I20260811 22:13:49.780635 19951 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:49.796913 19951 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:49.800607 19951 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:49.800835 19951 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:49.801585 19951 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:49.802496 19951 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:49.802570 19951 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:49.802644 19951 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:49.802699 19951 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:49.812427 19951 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:46563
I20260811 22:13:49.812453 20101 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:46563 every 8 connection(s)
I20260811 22:13:49.813452 19951 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:13:49.816006 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 19951
I20260811 22:13:49.816188 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:13:49.816597 28862 external_mini_cluster.cc:1428] Setting key 7ec7ac6825c6bc54f292a34a60363f0c
Aug 11 22:13:49 dist-test-slave-vxj2 krb5kdc[19507](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486429, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.131@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:13:49.829124 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.815417 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:58443 (local address 127.28.47.190:33061)
0811 22:13:49.815609 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:49.815614 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:49.816487 (+ 873us) server_negotiation.cc:407] Connection header received
0811 22:13:49.818094 (+ 1607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:49.818098 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:49.818152 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:49.818250 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:49.819688 (+ 1438us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.820233 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.821597 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.821825 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.821951 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.824740 (+ 2789us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:49.824743 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:49.824746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:49.824772 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:49.826457 (+ 1685us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:49.827109 (+ 652us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:49.827112 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:49.827114 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:49.827204 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:49.827744 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:49.827751 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:49.827753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:49.827979 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:49.828098 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:49.828754 (+ 656us) server_negotiation.cc:299] Negotiation successful
0811 22:13:49.828937 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260811 22:13:49.830267 20104 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.815807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:33061 (local address 127.28.47.131:58443)
0811 22:13:49.816299 (+ 492us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:49.816364 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:49.817810 (+ 1446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:49.818473 (+ 663us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:49.818482 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:49.819013 (+ 531us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:49.819544 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.819557 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.820379 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.820382 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.821455 (+ 1073us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.821467 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.822031 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.822034 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.822186 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.822889 (+ 703us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:13:49.822907 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:49.824543 (+ 1636us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:49.826597 (+ 2054us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:49.826603 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:49.826621 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:49.826996 (+ 375us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:49.827306 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:13:49.827309 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:13:49.827311 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:13:49.827483 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:13:49.828100 (+ 617us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:49.828110 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:49.828495 (+ 385us) client_negotiation.cc:769] Sending connection context
0811 22:13:49.828708 (+ 213us) client_negotiation.cc:240] Negotiation successful
0811 22:13:49.829083 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":290,"thread_start_us":147,"threads_started":1}
I20260811 22:13:49.831701 20102 heartbeater.cc:347] Connected to a master server at 127.28.47.190:33061
I20260811 22:13:49.831975 20102 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:49.832482 20102 heartbeater.cc:511] Master 127.28.47.190:33061 requested a full tablet report, sending...
I20260811 22:13:49.833674 19562 ts_manager.cc:194] Registered new tserver with Master: 85e229dd08af426f9846f78f39196449 (127.28.47.131:46563)
I20260811 22:13:49.834311 19562 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.28.47.131@KRBTEST.COM'} at 127.28.47.131:58443
I20260811 22:13:49.843537 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:13:49.854326 19615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.846013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50756 (local address 127.28.47.190:33061)
0811 22:13:49.846176 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:49.846181 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:49.846304 (+ 123us) server_negotiation.cc:407] Connection header received
0811 22:13:49.846497 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:49.846501 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:49.846555 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:49.846655 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:49.847496 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.848033 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.848903 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.849132 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.849317 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.850879 (+ 1562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:49.850889 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:49.850892 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:49.850921 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:49.852433 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:49.852940 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:49.852946 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:49.852948 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:49.853039 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:49.853353 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:49.853357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:49.853359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:49.853603 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:49.853753 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:49.854010 (+ 257us) server_negotiation.cc:299] Negotiation successful
0811 22:13:49.854139 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:13:49.857892 19562 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50756:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:13:49.861341 19562 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:13:49.877070 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.871005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58758 (local address 127.28.47.129:34073)
0811 22:13:49.871719 (+ 714us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:49.871727 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:49.871754 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:13:49.871838 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:49.871847 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:49.872095 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:49.872263 (+ 168us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:49.872765 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.873926 (+ 1161us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.875454 (+ 1528us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.876288 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.876408 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.876570 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:49.876789 (+ 219us) server_negotiation.cc:299] Negotiation successful
0811 22:13:49.876910 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":596,"thread_start_us":105,"threads_started":1}
I20260811 22:13:49.877065 20116 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.870880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:34073 (local address 127.0.0.1:58758)
0811 22:13:49.871268 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:49.871318 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:49.871462 (+ 144us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:49.872468 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:49.872474 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:49.872499 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:49.872641 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.872649 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.874023 (+ 1374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.874028 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.875303 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.875314 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.876433 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.876437 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.876566 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.876614 (+ 48us) client_negotiation.cc:769] Sending connection context
0811 22:13:49.876741 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 22:13:49.876893 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"mutex_wait_us":4,"thread_start_us":131,"threads_started":1}
I20260811 22:13:49.877554 20117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.870958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:36597 (local address 127.0.0.1:60872)
0811 22:13:49.872150 (+ 1192us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:49.872166 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:49.872248 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:49.872975 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:49.872978 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:49.872999 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:49.873122 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.873130 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.874608 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.874612 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.875793 (+ 1181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.875804 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.877078 (+ 1274us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.877082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.877215 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.877243 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:13:49.877349 (+ 106us) client_negotiation.cc:240] Negotiation successful
0811 22:13:49.877424 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1121,"mutex_wait_us":28,"thread_start_us":104,"threads_started":1}
I20260811 22:13:49.877631 20120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.871759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60872 (local address 127.28.47.130:36597)
0811 22:13:49.872502 (+ 743us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:49.872515 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:49.872537 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:13:49.872610 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:49.872616 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:49.872815 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:49.872977 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:49.873268 (+ 291us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.874456 (+ 1188us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.875938 (+ 1482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.876770 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.876864 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.877013 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:49.877394 (+ 381us) server_negotiation.cc:299] Negotiation successful
0811 22:13:49.877495 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":636,"thread_start_us":112,"threads_started":1}
I20260811 22:13:49.877903 20119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.871700 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:46563 (local address 127.0.0.1:41382)
0811 22:13:49.872493 (+ 793us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:49.872507 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:49.872593 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:49.873487 (+ 894us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:49.873490 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:49.873511 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:49.873628 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.873633 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.874921 (+ 1288us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.874927 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.876097 (+ 1170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.876108 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.877449 (+ 1341us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.877455 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.877578 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.877600 (+ 22us) client_negotiation.cc:769] Sending connection context
0811 22:13:49.877665 (+ 65us) client_negotiation.cc:240] Negotiation successful
0811 22:13:49.877730 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":734,"mutex_wait_us":29,"thread_start_us":108,"threads_started":1}
I20260811 22:13:49.878281 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.871885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41382 (local address 127.28.47.131:46563)
0811 22:13:49.872976 (+ 1091us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:49.872988 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:49.873011 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:13:49.873087 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:49.873092 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:49.873337 (+ 245us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:49.873497 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:49.873759 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.874784 (+ 1025us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.876237 (+ 1453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.877074 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.877214 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.877367 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:49.878023 (+ 656us) server_negotiation.cc:299] Negotiation successful
0811 22:13:49.878126 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":955,"thread_start_us":164,"threads_started":1}
I20260811 22:13:49.881028 19714 tablet_service.cc:1467] Processing CreateTablet for tablet c1ffcdea653146b393b312d57152e3cb (DEFAULT_TABLE table=test-table [id=15e232df58a746b7b55ce69d6da9a5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:49.881033 19873 tablet_service.cc:1467] Processing CreateTablet for tablet c1ffcdea653146b393b312d57152e3cb (DEFAULT_TABLE table=test-table [id=15e232df58a746b7b55ce69d6da9a5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:49.881006 20027 tablet_service.cc:1467] Processing CreateTablet for tablet c1ffcdea653146b393b312d57152e3cb (DEFAULT_TABLE table=test-table [id=15e232df58a746b7b55ce69d6da9a5ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:49.882182 20027 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ffcdea653146b393b312d57152e3cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:49.882189 19714 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ffcdea653146b393b312d57152e3cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:49.882189 19873 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet c1ffcdea653146b393b312d57152e3cb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:49.888863 20122 tablet_bootstrap.cc:492] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449: Bootstrap starting.
I20260811 22:13:49.890347 20123 tablet_bootstrap.cc:492] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0: Bootstrap starting.
I20260811 22:13:49.892153 20123 tablet_bootstrap.cc:654] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:49.892154 20122 tablet_bootstrap.cc:654] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:49.892941 20123 log.cc:824] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:49.893297 20122 log.cc:824] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:49.893738 20124 tablet_bootstrap.cc:492] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b: Bootstrap starting.
I20260811 22:13:49.894995 20123 tablet_bootstrap.cc:492] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0: No bootstrap required, opened a new log
I20260811 22:13:49.895231 20123 ts_tablet_manager.cc:1397] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260811 22:13:49.895629 20122 tablet_bootstrap.cc:492] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449: No bootstrap required, opened a new log
I20260811 22:13:49.895821 20122 ts_tablet_manager.cc:1397] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.003s
I20260811 22:13:49.896100 20124 tablet_bootstrap.cc:654] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:49.897399 20124 log.cc:824] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:49.899353 20123 raft_consensus.cc:348] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0370ed960dca45abac71c5f40fc46cb0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34073 } } peers { permanent_uuid: "e63e583098b1429ea959258ae35c674b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 36597 } } peers { permanent_uuid: "85e229dd08af426f9846f78f39196449" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46563 } }
I20260811 22:13:49.899607 20124 tablet_bootstrap.cc:492] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b: No bootstrap required, opened a new log
I20260811 22:13:49.899646 20123 raft_consensus.cc:374] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:49.899704 20123 raft_consensus.cc:729] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0370ed960dca45abac71c5f40fc46cb0, State: Initialized, Role: FOLLOWER
I20260811 22:13:49.899796 20124 ts_tablet_manager.cc:1397] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260811 22:13:49.899821 20122 raft_consensus.cc:348] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0370ed960dca45abac71c5f40fc46cb0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34073 } } peers { permanent_uuid: "e63e583098b1429ea959258ae35c674b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 36597 } } peers { permanent_uuid: "85e229dd08af426f9846f78f39196449" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46563 } }
I20260811 22:13:49.900090 20122 raft_consensus.cc:374] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:49.900163 20122 raft_consensus.cc:729] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 85e229dd08af426f9846f78f39196449, State: Initialized, Role: FOLLOWER
I20260811 22:13:49.900297 20123 consensus_queue.cc:260] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0370ed960dca45abac71c5f40fc46cb0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34073 } } peers { permanent_uuid: "e63e583098b1429ea959258ae35c674b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 36597 } } peers { permanent_uuid: "85e229dd08af426f9846f78f39196449" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46563 } }
I20260811 22:13:49.900597 20122 consensus_queue.cc:260] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0370ed960dca45abac71c5f40fc46cb0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34073 } } peers { permanent_uuid: "e63e583098b1429ea959258ae35c674b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 36597 } } peers { permanent_uuid: "85e229dd08af426f9846f78f39196449" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46563 } }
I20260811 22:13:49.901139 19789 heartbeater.cc:503] Master 127.28.47.190:33061 was elected leader, sending a full tablet report...
I20260811 22:13:49.901665 20122 ts_tablet_manager.cc:1428] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260811 22:13:49.902043 20102 heartbeater.cc:503] Master 127.28.47.190:33061 was elected leader, sending a full tablet report...
I20260811 22:13:49.902628 20123 ts_tablet_manager.cc:1428] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260811 22:13:49.904125 20124 raft_consensus.cc:348] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0370ed960dca45abac71c5f40fc46cb0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34073 } } peers { permanent_uuid: "e63e583098b1429ea959258ae35c674b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 36597 } } peers { permanent_uuid: "85e229dd08af426f9846f78f39196449" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46563 } }
I20260811 22:13:49.904444 20124 raft_consensus.cc:374] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:49.904506 20124 raft_consensus.cc:729] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e63e583098b1429ea959258ae35c674b, State: Initialized, Role: FOLLOWER
I20260811 22:13:49.905011 20124 consensus_queue.cc:260] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0370ed960dca45abac71c5f40fc46cb0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34073 } } peers { permanent_uuid: "e63e583098b1429ea959258ae35c674b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 36597 } } peers { permanent_uuid: "85e229dd08af426f9846f78f39196449" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46563 } }
I20260811 22:13:49.905926 19946 heartbeater.cc:503] Master 127.28.47.190:33061 was elected leader, sending a full tablet report...
I20260811 22:13:49.907341 20124 ts_tablet_manager.cc:1428] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b: Time spent starting tablet: real 0.007s user 0.003s sys 0.004s
I20260811 22:13:49.937319 20128 raft_consensus.cc:482] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:13:49.937534 20128 raft_consensus.cc:504] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0370ed960dca45abac71c5f40fc46cb0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34073 } } peers { permanent_uuid: "e63e583098b1429ea959258ae35c674b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 36597 } } peers { permanent_uuid: "85e229dd08af426f9846f78f39196449" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46563 } }
I20260811 22:13:49.938690 20128 leader_election.cc:288] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e63e583098b1429ea959258ae35c674b (127.28.47.130:36597), 85e229dd08af426f9846f78f39196449 (127.28.47.131:46563)
I20260811 22:13:49.944707 20121 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.939120 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:36413 (local address 127.28.47.131:46563)
0811 22:13:49.939268 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:49.939272 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:49.939688 (+ 416us) server_negotiation.cc:407] Connection header received
0811 22:13:49.939787 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:49.939790 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:49.939855 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:49.939954 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:49.940440 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.941323 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.943389 (+ 2066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.944035 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.944198 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.944405 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:49.944464 (+ 59us) server_negotiation.cc:299] Negotiation successful
0811 22:13:49.944565 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:13:49.944769 20132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.939009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:46563 (local address 127.28.47.129:36413)
0811 22:13:49.939580 (+ 571us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:49.939593 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:49.939691 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:49.940009 (+ 318us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:49.940012 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:49.940041 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:49.940281 (+ 240us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.940287 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.941472 (+ 1185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.941476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.943208 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.943223 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.944233 (+ 1010us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.944237 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.944347 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.944374 (+ 27us) client_negotiation.cc:769] Sending connection context
0811 22:13:49.944438 (+ 64us) client_negotiation.cc:240] Negotiation successful
0811 22:13:49.944513 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"mutex_wait_us":546,"spinlock_wait_cycles":11392,"thread_start_us":95,"threads_started":1}
I20260811 22:13:49.945525 20120 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.939126 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:47589 (local address 127.28.47.130:36597)
0811 22:13:49.939293 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:49.939298 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:49.939679 (+ 381us) server_negotiation.cc:407] Connection header received
0811 22:13:49.939856 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:49.939859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:49.939920 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:49.940001 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:49.940440 (+ 439us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.941336 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.943389 (+ 2053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.944083 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.944215 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.944405 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:49.945245 (+ 840us) server_negotiation.cc:299] Negotiation successful
0811 22:13:49.945375 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260811 22:13:49.945576 20131 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.939009 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:36597 (local address 127.28.47.129:47589)
0811 22:13:49.939521 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:49.939539 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:49.939687 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:49.940016 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:49.940018 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:49.940046 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:49.940273 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.940282 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.941472 (+ 1190us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.941476 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.943208 (+ 1732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:49.943223 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.944498 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.944503 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:49.944611 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.944666 (+ 55us) client_negotiation.cc:769] Sending connection context
0811 22:13:49.945345 (+ 679us) client_negotiation.cc:240] Negotiation successful
0811 22:13:49.945431 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":390,"mutex_wait_us":202,"spinlock_wait_cycles":4352,"thread_start_us":148,"threads_started":1}
I20260811 22:13:49.945700 20048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ffcdea653146b393b312d57152e3cb" candidate_uuid: "0370ed960dca45abac71c5f40fc46cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e229dd08af426f9846f78f39196449" is_pre_election: true
I20260811 22:13:49.946099 20048 raft_consensus.cc:2454] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0370ed960dca45abac71c5f40fc46cb0 in term 0.
I20260811 22:13:49.946393 19895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ffcdea653146b393b312d57152e3cb" candidate_uuid: "0370ed960dca45abac71c5f40fc46cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63e583098b1429ea959258ae35c674b" is_pre_election: true
I20260811 22:13:49.946727 19895 raft_consensus.cc:2454] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 0370ed960dca45abac71c5f40fc46cb0 in term 0.
I20260811 22:13:49.946746 19663 leader_election.cc:302] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0370ed960dca45abac71c5f40fc46cb0, 85e229dd08af426f9846f78f39196449; no voters:
I20260811 22:13:49.947109 20128 raft_consensus.cc:2781] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:13:49.947217 20128 raft_consensus.cc:482] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:13:49.947263 20128 raft_consensus.cc:3037] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:49.948424 20128 raft_consensus.cc:504] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0370ed960dca45abac71c5f40fc46cb0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34073 } } peers { permanent_uuid: "e63e583098b1429ea959258ae35c674b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 36597 } } peers { permanent_uuid: "85e229dd08af426f9846f78f39196449" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46563 } }
I20260811 22:13:49.948809 20128 leader_election.cc:288] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [CANDIDATE]: Term 1 election: Requested vote from peers e63e583098b1429ea959258ae35c674b (127.28.47.130:36597), 85e229dd08af426f9846f78f39196449 (127.28.47.131:46563)
I20260811 22:13:49.949232 19895 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ffcdea653146b393b312d57152e3cb" candidate_uuid: "0370ed960dca45abac71c5f40fc46cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e63e583098b1429ea959258ae35c674b"
I20260811 22:13:49.949292 20048 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "c1ffcdea653146b393b312d57152e3cb" candidate_uuid: "0370ed960dca45abac71c5f40fc46cb0" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "85e229dd08af426f9846f78f39196449"
I20260811 22:13:49.949352 19895 raft_consensus.cc:3037] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:49.949456 20048 raft_consensus.cc:3037] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:49.950480 19895 raft_consensus.cc:2454] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0370ed960dca45abac71c5f40fc46cb0 in term 1.
I20260811 22:13:49.950670 20048 raft_consensus.cc:2454] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0370ed960dca45abac71c5f40fc46cb0 in term 1.
I20260811 22:13:49.950935 19662 leader_election.cc:302] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0370ed960dca45abac71c5f40fc46cb0, e63e583098b1429ea959258ae35c674b; no voters:
I20260811 22:13:49.951138 20128 raft_consensus.cc:2781] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:49.951387 20128 raft_consensus.cc:686] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [term 1 LEADER]: Becoming Leader. State: Replica: 0370ed960dca45abac71c5f40fc46cb0, State: Running, Role: LEADER
I20260811 22:13:49.951795 20128 consensus_queue.cc:237] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0370ed960dca45abac71c5f40fc46cb0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34073 } } peers { permanent_uuid: "e63e583098b1429ea959258ae35c674b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 36597 } } peers { permanent_uuid: "85e229dd08af426f9846f78f39196449" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46563 } }
I20260811 22:13:49.955405 19561 catalog_manager.cc:5495] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 reported cstate change: term changed from 0 to 1, leader changed from <none> to 0370ed960dca45abac71c5f40fc46cb0 (127.28.47.129). New cstate: current_term: 1 leader_uuid: "0370ed960dca45abac71c5f40fc46cb0" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0370ed960dca45abac71c5f40fc46cb0" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 34073 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e63e583098b1429ea959258ae35c674b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 36597 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "85e229dd08af426f9846f78f39196449" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46563 } health_report { overall_health: UNKNOWN } } }
I20260811 22:13:49.980234 20118 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:49.976566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58772 (local address 127.28.47.129:34073)
0811 22:13:49.976705 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:49.976708 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:49.976724 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:13:49.976898 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:49.976901 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:49.976965 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:49.977057 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:13:49.977407 (+ 350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.977935 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.978811 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:49.979041 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:49.979118 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:49.979393 (+ 275us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:13:49.979861 (+ 468us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:13:49.979960 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:49.980060 (+ 100us) server_negotiation.cc:299] Negotiation successful
0811 22:13:49.980114 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:13:49.987058 19895 raft_consensus.cc:1261] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b [term 1 FOLLOWER]: Refusing update from remote peer 0370ed960dca45abac71c5f40fc46cb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:49.987058 20048 raft_consensus.cc:1261] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449 [term 1 FOLLOWER]: Refusing update from remote peer 0370ed960dca45abac71c5f40fc46cb0: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:49.987979 20133 consensus_queue.cc:1035] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e63e583098b1429ea959258ae35c674b" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 36597 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:49.988191 20128 consensus_queue.cc:1035] T c1ffcdea653146b393b312d57152e3cb P 0370ed960dca45abac71c5f40fc46cb0 [LEADER]: Connected to new peer: Peer: permanent_uuid: "85e229dd08af426f9846f78f39196449" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 46563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:49.996007 20138 mvcc.cc:205] Tried to move back new op lower bound from 7317448417218129920 to 7317448417085583360. Current Snapshot: MvccSnapshot[applied={T|T < 7317448417218129920}]
I20260811 22:13:49.998100 20140 mvcc.cc:205] Tried to move back new op lower bound from 7317448417218129920 to 7317448417085583360. Current Snapshot: MvccSnapshot[applied={T|T < 7317448417218129920}]
I20260811 22:13:50.008602 19562 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:50.008814 19562 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50756:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:50.011838 19562 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 10862fd4e8f8449887e228622a055e08: Sending DeleteTablet for 3 replicas of tablet c1ffcdea653146b393b312d57152e3cb
I20260811 22:13:50.012867 19714 tablet_service.cc:1514] Processing DeleteTablet for tablet c1ffcdea653146b393b312d57152e3cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:50 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:58758
I20260811 22:13:50.013032 19873 tablet_service.cc:1514] Processing DeleteTablet for tablet c1ffcdea653146b393b312d57152e3cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:50 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:60872
I20260811 22:13:50.013247 20027 tablet_service.cc:1514] Processing DeleteTablet for tablet c1ffcdea653146b393b312d57152e3cb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:50 UTC) from {username='kudu', principal='kudu/127.28.47.190@KRBTEST.COM'} at 127.0.0.1:41382
I20260811 22:13:50.014050 20151 tablet_replica.cc:330] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449: stopping tablet replica
I20260811 22:13:50.014438 20151 raft_consensus.cc:2229] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:50.014802 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 19639
I20260811 22:13:50.014842 20151 raft_consensus.cc:2258] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:50.015574 20150 tablet_replica.cc:330] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b: stopping tablet replica
I20260811 22:13:50.015949 20150 raft_consensus.cc:2229] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:50.016289 20150 raft_consensus.cc:2258] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:50.018752 20151 ts_tablet_manager.cc:1905] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:50.018790 20150 ts_tablet_manager.cc:1905] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:50.022029 20150 ts_tablet_manager.cc:1918] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:50.022254 20150 log.cc:1196] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/c1ffcdea653146b393b312d57152e3cb
I20260811 22:13:50.022686 20150 ts_tablet_manager.cc:1939] T c1ffcdea653146b393b312d57152e3cb P e63e583098b1429ea959258ae35c674b: Deleting consensus metadata
I20260811 22:13:50.023104 20151 ts_tablet_manager.cc:1918] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:50.023209 20151 log.cc:1196] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/c1ffcdea653146b393b312d57152e3cb
I20260811 22:13:50.023540 20151 ts_tablet_manager.cc:1939] T c1ffcdea653146b393b312d57152e3cb P 85e229dd08af426f9846f78f39196449: Deleting consensus metadata
I20260811 22:13:50.024734 19546 catalog_manager.cc:4841] TS e63e583098b1429ea959258ae35c674b (127.28.47.130:36597): tablet c1ffcdea653146b393b312d57152e3cb (table test-table [id=15e232df58a746b7b55ce69d6da9a5ba]) successfully deleted
I20260811 22:13:50.026023 19547 catalog_manager.cc:4841] TS 85e229dd08af426f9846f78f39196449 (127.28.47.131:46563): tablet c1ffcdea653146b393b312d57152e3cb (table test-table [id=15e232df58a746b7b55ce69d6da9a5ba]) successfully deleted
W20260811 22:13:50.030489 19542 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:34073 (error 108)
I20260811 22:13:50.030640 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 19794
I20260811 22:13:50.036229 20119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:50.035831 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:34073 (local address 127.0.0.1:58788)
0811 22:13:50.036015 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:50.036140 (+ 125us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:34073: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":79}
W20260811 22:13:50.036443 19542 catalog_manager.cc:4584] TS 0370ed960dca45abac71c5f40fc46cb0 (127.28.47.129:34073): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet c1ffcdea653146b393b312d57152e3cb: Network error: Client connection negotiation failed: client connection to 127.28.47.129:34073: connect: Connection refused (error 111)
I20260811 22:13:50.041904 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 19951
I20260811 22:13:50.052860 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 19524
2026-08-11T22:13:50Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2559 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-11T22:13:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:50Z Disabled control of system clock
I20260811 22:13:50.088431 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:37243
--webserver_interface=127.28.47.190
--webserver_port=0
--builtin_ntp_servers=127.28.47.148:46395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:37243
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:13:50.213090 20160 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:50.213459 20160 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:50.213510 20160 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:50.217978 20160 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:50.218055 20160 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:50.218070 20160 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:50.218087 20160 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:50.218102 20160 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:50.223340 20160 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:46395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:37243
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:37243
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20160
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:50.224695 20160 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:50.226328 20160 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:50.235553 20167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:50.235553 20169 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:50.235579 20166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:50.236018 20160 server_base.cc:1034] running on GCE node
I20260811 22:13:50.236649 20160 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:50.237985 20160 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:50.239178 20160 hybrid_clock.cc:648] HybridClock initialized: now 1786486430239154 us; error 40 us; skew 500 ppm
I20260811 22:13:50.241469 20160 webserver.cc:460] Webserver started at http://127.28.47.190:39141/ using document root <none> and password file <none>
I20260811 22:13:50.242125 20160 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:50.242195 20160 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:50.242436 20160 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:50.244661 20160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "b8baba57abc043a4928e3452d7ca5f85"
format_stamp: "Formatted at 2026-08-11 22:13:50 on dist-test-slave-vxj2"
server_key: "72644d2c43185313b64a8b4743c8e785"
server_key_iv: "5f6e2c17cf42d08d83a3ee0256854c9c"
server_key_version: "encryptionkey@0"
I20260811 22:13:50.245355 20160 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "b8baba57abc043a4928e3452d7ca5f85"
format_stamp: "Formatted at 2026-08-11 22:13:50 on dist-test-slave-vxj2"
server_key: "72644d2c43185313b64a8b4743c8e785"
server_key_iv: "5f6e2c17cf42d08d83a3ee0256854c9c"
server_key_version: "encryptionkey@0"
I20260811 22:13:50.249609 20160 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:13:50.252463 20178 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:50.253988 20160 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:50.254163 20160 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "b8baba57abc043a4928e3452d7ca5f85"
format_stamp: "Formatted at 2026-08-11 22:13:50 on dist-test-slave-vxj2"
server_key: "72644d2c43185313b64a8b4743c8e785"
server_key_iv: "5f6e2c17cf42d08d83a3ee0256854c9c"
server_key_version: "encryptionkey@0"
I20260811 22:13:50.254304 20160 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:50.276412 20160 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:50.277289 20160 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:50.277518 20160 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:50.285441 20160 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:37243
I20260811 22:13:50.285506 20236 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:37243 every 8 connection(s)
I20260811 22:13:50.286561 20160 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:50.289994 20237 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:50.296216 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 20160
I20260811 22:13:50.296283 20237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85: Bootstrap starting.
I20260811 22:13:50.296396 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:13:50.296682 28862 external_mini_cluster.cc:1428] Setting key 584e6706693279399c60a16d69e2cdaf
I20260811 22:13:50.299049 20237 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:50.299875 20237 log.cc:824] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:50.302253 20237 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85: No bootstrap required, opened a new log
I20260811 22:13:50.306432 20237 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8baba57abc043a4928e3452d7ca5f85" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 37243 } }
I20260811 22:13:50.306743 20237 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:50.306824 20237 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8baba57abc043a4928e3452d7ca5f85, State: Initialized, Role: FOLLOWER
I20260811 22:13:50.307343 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:50.298192 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38752 (local address 127.28.47.190:37243)
0811 22:13:50.298981 (+ 789us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:50.299006 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:50.299062 (+ 56us) server_negotiation.cc:407] Connection header received
0811 22:13:50.299817 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:50.299837 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:50.300366 (+ 529us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:50.300779 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:50.301221 (+ 442us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.302255 (+ 1034us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.303186 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.303893 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.304093 (+ 200us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:50.305130 (+ 1037us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:50.305136 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:50.305199 (+ 63us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:50.305657 (+ 458us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:50.305723 (+ 66us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:50.305747 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:50.305885 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:50.306053 (+ 168us) server_negotiation.cc:299] Negotiation successful
0811 22:13:50.306368 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":157,"threads_started":1}
I20260811 22:13:50.307325 20237 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8baba57abc043a4928e3452d7ca5f85" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 37243 } }
I20260811 22:13:50.307497 20237 raft_consensus.cc:388] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:50.307595 20237 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:50.307708 20237 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:50.308969 20237 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8baba57abc043a4928e3452d7ca5f85" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 37243 } }
I20260811 22:13:50.309397 20237 leader_election.cc:302] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b8baba57abc043a4928e3452d7ca5f85; no voters:
I20260811 22:13:50.309775 20237 leader_election.cc:288] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:50.309962 20243 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:50.310216 20243 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [term 1 LEADER]: Becoming Leader. State: Replica: b8baba57abc043a4928e3452d7ca5f85, State: Running, Role: LEADER
I20260811 22:13:50.310563 20243 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8baba57abc043a4928e3452d7ca5f85" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 37243 } }
I20260811 22:13:50.311152 20237 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:50.312419 20245 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8baba57abc043a4928e3452d7ca5f85. Latest consensus state: current_term: 1 leader_uuid: "b8baba57abc043a4928e3452d7ca5f85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8baba57abc043a4928e3452d7ca5f85" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 37243 } } }
I20260811 22:13:50.312584 20245 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:50.312868 20244 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b8baba57abc043a4928e3452d7ca5f85" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b8baba57abc043a4928e3452d7ca5f85" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 37243 } } }
I20260811 22:13:50.312980 20244 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:50.313110 20254 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:50.316293 20254 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:50.322816 20254 catalog_manager.cc:1347] Generated new cluster ID: 2994f7f53055422f985bf43ac2abed18
I20260811 22:13:50.322930 20254 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:50.343230 20254 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:50.344394 20254 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:50.351435 20254 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85: Generated new TSK 0
I20260811 22:13:50.352254 20254 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:13:50.359548 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.129:0
--local_ip_for_outbound_sockets=127.28.47.129
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:37243
--builtin_ntp_servers=127.28.47.148:46395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260811 22:13:50.482105 20264 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:50.482434 20264 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:50.482478 20264 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:50.486871 20264 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:50.486955 20264 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:50.487016 20264 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.129
I20260811 22:13:50.492264 20264 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:46395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.28.47.129
--webserver_port=0
--tserver_master_addrs=127.28.47.190:37243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20264
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.129
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:50.493654 20264 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:50.495237 20264 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:50.506047 20272 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:50.506047 20269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:50.506155 20270 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:50.506560 20264 server_base.cc:1034] running on GCE node
I20260811 22:13:50.507064 20264 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:50.507843 20264 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:50.509057 20264 hybrid_clock.cc:648] HybridClock initialized: now 1786486430509017 us; error 58 us; skew 500 ppm
I20260811 22:13:50.511464 20264 webserver.cc:460] Webserver started at http://127.28.47.129:43361/ using document root <none> and password file <none>
I20260811 22:13:50.512208 20264 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:50.512285 20264 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:50.512476 20264 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:50.514937 20264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/data/instance:
uuid: "59869f03fcdc49d08b32702db054cfdb"
format_stamp: "Formatted at 2026-08-11 22:13:50 on dist-test-slave-vxj2"
server_key: "577ef6bf7eea41b0f19390c985def53f"
server_key_iv: "00d576a525735133afb376c3b1813a4e"
server_key_version: "encryptionkey@0"
I20260811 22:13:50.515569 20264 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance:
uuid: "59869f03fcdc49d08b32702db054cfdb"
format_stamp: "Formatted at 2026-08-11 22:13:50 on dist-test-slave-vxj2"
server_key: "577ef6bf7eea41b0f19390c985def53f"
server_key_iv: "00d576a525735133afb376c3b1813a4e"
server_key_version: "encryptionkey@0"
I20260811 22:13:50.519807 20264 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:13:50.522719 20280 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:50.524111 20264 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:13:50.524281 20264 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/wal
uuid: "59869f03fcdc49d08b32702db054cfdb"
format_stamp: "Formatted at 2026-08-11 22:13:50 on dist-test-slave-vxj2"
server_key: "577ef6bf7eea41b0f19390c985def53f"
server_key_iv: "00d576a525735133afb376c3b1813a4e"
server_key_version: "encryptionkey@0"
I20260811 22:13:50.524420 20264 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:50.540177 20264 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:50.541042 20264 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:50.541306 20264 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:50.542023 20264 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:50.543097 20264 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:50.543171 20264 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:50.543246 20264 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:50.543299 20264 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:50.553504 20264 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.129:39863
I20260811 22:13:50.553531 20417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.129:39863 every 8 connection(s)
I20260811 22:13:50.554564 20264 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/data/info.pb
I20260811 22:13:50.558064 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 20264
I20260811 22:13:50.558218 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-0/wal/instance
I20260811 22:13:50.558501 28862 external_mini_cluster.cc:1428] Setting key 7d54dc9554c06b9adbb9bae3aff4df15
I20260811 22:13:50.560775 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.130:0
--local_ip_for_outbound_sockets=127.28.47.130
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:37243
--builtin_ntp_servers=127.28.47.148:46395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:13:50.566540 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:50.556704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:40723 (local address 127.28.47.190:37243)
0811 22:13:50.556938 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:50.556942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:50.558381 (+ 1439us) server_negotiation.cc:407] Connection header received
0811 22:13:50.560358 (+ 1977us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:50.560362 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:50.560424 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:50.560535 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:50.561804 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.562375 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.563718 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.564185 (+ 467us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.564335 (+ 150us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:50.565823 (+ 1488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:50.565830 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:50.565835 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:50.565858 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:50.565890 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:50.565903 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:50.566000 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:50.566184 (+ 184us) server_negotiation.cc:299] Negotiation successful
0811 22:13:50.566278 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119}
I20260811 22:13:50.567410 20420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:50.557241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:37243 (local address 127.28.47.129:40723)
0811 22:13:50.558179 (+ 938us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:50.558242 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:50.560063 (+ 1821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:50.560748 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:50.560800 (+ 52us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:50.560948 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:50.561638 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:50.561655 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.562533 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.562537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:50.563547 (+ 1010us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:50.563563 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.564395 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.564399 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:50.564614 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:50.565475 (+ 861us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:50.565476 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:50.565535 (+ 59us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:50.565619 (+ 84us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:50.566007 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:50.566016 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:50.566021 (+ 5us) client_negotiation.cc:769] Sending connection context
0811 22:13:50.566126 (+ 105us) client_negotiation.cc:240] Negotiation successful
0811 22:13:50.566421 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":301,"thread_start_us":134,"threads_started":1}
I20260811 22:13:50.569005 20418 heartbeater.cc:347] Connected to a master server at 127.28.47.190:37243
I20260811 22:13:50.569440 20418 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:50.570294 20418 heartbeater.cc:511] Master 127.28.47.190:37243 requested a full tablet report, sending...
I20260811 22:13:50.572340 20197 ts_manager.cc:194] Registered new tserver with Master: 59869f03fcdc49d08b32702db054cfdb (127.28.47.129:39863)
I20260811 22:13:50.573729 20197 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.47.129:40723
W20260811 22:13:50.683223 20421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:50.683555 20421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:50.683598 20421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:50.688048 20421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:50.688130 20421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:50.688192 20421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.130
I20260811 22:13:50.693529 20421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:46395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.28.47.130
--webserver_port=0
--tserver_master_addrs=127.28.47.190:37243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.130
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:50.694818 20421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:50.696398 20421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:50.707100 20429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:50.707100 20427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:50.707125 20426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:50.707547 20421 server_base.cc:1034] running on GCE node
I20260811 22:13:50.708056 20421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:50.708822 20421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:50.710067 20421 hybrid_clock.cc:648] HybridClock initialized: now 1786486430710016 us; error 72 us; skew 500 ppm
I20260811 22:13:50.712601 20421 webserver.cc:460] Webserver started at http://127.28.47.130:43577/ using document root <none> and password file <none>
I20260811 22:13:50.713376 20421 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:50.713488 20421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:50.713739 20421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:50.716043 20421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/data/instance:
uuid: "afb8b849f1a34d7999c31ae29056addb"
format_stamp: "Formatted at 2026-08-11 22:13:50 on dist-test-slave-vxj2"
server_key: "7e0daaef08ebcbf6aab417dc9ccee7b6"
server_key_iv: "70060087a0486f40da91002642a9455e"
server_key_version: "encryptionkey@0"
I20260811 22:13:50.716619 20421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance:
uuid: "afb8b849f1a34d7999c31ae29056addb"
format_stamp: "Formatted at 2026-08-11 22:13:50 on dist-test-slave-vxj2"
server_key: "7e0daaef08ebcbf6aab417dc9ccee7b6"
server_key_iv: "70060087a0486f40da91002642a9455e"
server_key_version: "encryptionkey@0"
I20260811 22:13:50.720722 20421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:13:50.723779 20435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:50.725592 20421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:50.725769 20421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/wal
uuid: "afb8b849f1a34d7999c31ae29056addb"
format_stamp: "Formatted at 2026-08-11 22:13:50 on dist-test-slave-vxj2"
server_key: "7e0daaef08ebcbf6aab417dc9ccee7b6"
server_key_iv: "70060087a0486f40da91002642a9455e"
server_key_version: "encryptionkey@0"
I20260811 22:13:50.725925 20421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:50.738610 20421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:50.739441 20421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:50.739668 20421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:50.740365 20421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:50.741503 20421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:50.741578 20421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:50.741657 20421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:50.741701 20421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:50.751482 20421 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.130:39347
I20260811 22:13:50.751502 20569 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.130:39347 every 8 connection(s)
I20260811 22:13:50.752522 20421 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/data/info.pb
I20260811 22:13:50.759249 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 20421
I20260811 22:13:50.759413 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/wal/instance
I20260811 22:13:50.759713 28862 external_mini_cluster.cc:1428] Setting key 542780c522c1e1dc809e3df6b6e4cd9c
I20260811 22:13:50.762266 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.28.47.131:0
--local_ip_for_outbound_sockets=127.28.47.131
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:37243
--builtin_ntp_servers=127.28.47.148:46395
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260811 22:13:50.764227 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:50.754601 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.130:58135 (local address 127.28.47.190:37243)
0811 22:13:50.754771 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:50.754776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:50.756058 (+ 1282us) server_negotiation.cc:407] Connection header received
0811 22:13:50.757545 (+ 1487us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:50.757548 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:50.757602 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:50.757686 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:50.758957 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.759782 (+ 825us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.761578 (+ 1796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.761897 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.762028 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:50.763545 (+ 1517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:50.763553 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:50.763559 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:50.763579 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:50.763608 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:50.763621 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:50.763706 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:50.763996 (+ 290us) server_negotiation.cc:299] Negotiation successful
0811 22:13:50.764087 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:13:50.765714 20573 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:50.755087 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:37243 (local address 127.28.47.130:58135)
0811 22:13:50.755884 (+ 797us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:50.755929 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:50.757319 (+ 1390us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:50.758013 (+ 694us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:50.758021 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:50.758111 (+ 90us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:50.758757 (+ 646us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:50.758781 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.759974 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.759977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:50.761457 (+ 1480us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:50.761472 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.762088 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.762091 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:50.762391 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:50.763215 (+ 824us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:50.763217 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:50.763265 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:50.763348 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:50.763774 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:50.763785 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:50.763791 (+ 6us) client_negotiation.cc:769] Sending connection context
0811 22:13:50.763982 (+ 191us) client_negotiation.cc:240] Negotiation successful
0811 22:13:50.764398 (+ 416us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":124,"threads_started":1}
I20260811 22:13:50.767074 20570 heartbeater.cc:347] Connected to a master server at 127.28.47.190:37243
I20260811 22:13:50.767364 20570 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:50.767980 20570 heartbeater.cc:511] Master 127.28.47.190:37243 requested a full tablet report, sending...
I20260811 22:13:50.769251 20197 ts_manager.cc:194] Registered new tserver with Master: afb8b849f1a34d7999c31ae29056addb (127.28.47.130:39347)
I20260811 22:13:50.769867 20197 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.47.130:58135
W20260811 22:13:50.884943 20574 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:50.885279 20574 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:50.885370 20574 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:50.889674 20574 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:50.889828 20574 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:50.889945 20574 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.28.47.131
I20260811 22:13:50.895098 20574 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:46395
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.28.47.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.28.47.131
--webserver_port=0
--tserver_master_addrs=127.28.47.190:37243
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20574
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.28.47.131
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:50.896294 20574 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:50.897766 20574 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:50.908555 20580 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:50.908684 20574 server_base.cc:1034] running on GCE node
W20260811 22:13:50.908551 20582 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:50.908551 20579 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:50.909473 20574 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:50.910312 20574 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:50.911531 20574 hybrid_clock.cc:648] HybridClock initialized: now 1786486430911472 us; error 73 us; skew 500 ppm
I20260811 22:13:50.914021 20574 webserver.cc:460] Webserver started at http://127.28.47.131:37379/ using document root <none> and password file <none>
I20260811 22:13:50.914700 20574 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:50.914806 20574 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:50.915069 20574 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:50.917438 20574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/data/instance:
uuid: "ffebb4067fb746998d57c421e9304e9b"
format_stamp: "Formatted at 2026-08-11 22:13:50 on dist-test-slave-vxj2"
server_key: "c8d334e8bf0f0b07e0ead4df98200aa5"
server_key_iv: "eeb8f4ed846378b56ba99cbc307789e2"
server_key_version: "encryptionkey@0"
I20260811 22:13:50.918053 20574 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance:
uuid: "ffebb4067fb746998d57c421e9304e9b"
format_stamp: "Formatted at 2026-08-11 22:13:50 on dist-test-slave-vxj2"
server_key: "c8d334e8bf0f0b07e0ead4df98200aa5"
server_key_iv: "eeb8f4ed846378b56ba99cbc307789e2"
server_key_version: "encryptionkey@0"
I20260811 22:13:50.922286 20574 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:13:50.925104 20589 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:50.926571 20574 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:50.926756 20574 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/wal
uuid: "ffebb4067fb746998d57c421e9304e9b"
format_stamp: "Formatted at 2026-08-11 22:13:50 on dist-test-slave-vxj2"
server_key: "c8d334e8bf0f0b07e0ead4df98200aa5"
server_key_iv: "eeb8f4ed846378b56ba99cbc307789e2"
server_key_version: "encryptionkey@0"
I20260811 22:13:50.926899 20574 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:50.968505 20574 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:50.969396 20574 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:50.969640 20574 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:50.970342 20574 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260811 22:13:50.971436 20574 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260811 22:13:50.971511 20574 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:50.971581 20574 ts_tablet_manager.cc:610] Registered 0 tablets
I20260811 22:13:50.971630 20574 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:50.981412 20574 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.131:33693
I20260811 22:13:50.981550 20722 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.131:33693 every 8 connection(s)
I20260811 22:13:50.982467 20574 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/data/info.pb
I20260811 22:13:50.992067 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 20574
I20260811 22:13:50.992223 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/wal/instance
I20260811 22:13:50.992550 28862 external_mini_cluster.cc:1428] Setting key e2f91ec29525212dcac0fef5b20a208f
I20260811 22:13:50.993079 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:50.984543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.131:37721 (local address 127.28.47.190:37243)
0811 22:13:50.984699 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:50.984703 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:50.985889 (+ 1186us) server_negotiation.cc:407] Connection header received
0811 22:13:50.987371 (+ 1482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:50.987374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:50.987427 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:50.987534 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:50.988588 (+ 1054us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.989227 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.990446 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.990727 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.990829 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:50.992456 (+ 1627us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:50.992460 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:50.992462 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:50.992482 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:50.992505 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:50.992514 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:50.992586 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:50.992828 (+ 242us) server_negotiation.cc:299] Negotiation successful
0811 22:13:50.992904 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:13:50.994480 20725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:50.984959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:37243 (local address 127.28.47.131:37721)
0811 22:13:50.985704 (+ 745us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:50.985746 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:50.987123 (+ 1377us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:50.987720 (+ 597us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:50.987728 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:50.987855 (+ 127us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:13:50.988394 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:50.988408 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.989366 (+ 958us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.989369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:50.990313 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:50.990323 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:50.991074 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:50.991079 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:50.991256 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:50.991993 (+ 737us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0811 22:13:50.991996 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:13:50.992063 (+ 67us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0811 22:13:50.992167 (+ 104us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:13:50.992639 (+ 472us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:13:50.992651 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:13:50.992660 (+ 9us) client_negotiation.cc:769] Sending connection context
0811 22:13:50.992863 (+ 203us) client_negotiation.cc:240] Negotiation successful
0811 22:13:50.993255 (+ 392us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":159,"threads_started":1}
I20260811 22:13:50.995669 20723 heartbeater.cc:347] Connected to a master server at 127.28.47.190:37243
I20260811 22:13:50.995936 20723 heartbeater.cc:464] Registering TS with master...
I20260811 22:13:50.996528 20723 heartbeater.cc:511] Master 127.28.47.190:37243 requested a full tablet report, sending...
I20260811 22:13:50.997812 20197 ts_manager.cc:194] Registered new tserver with Master: ffebb4067fb746998d57c421e9304e9b (127.28.47.131:33693)
I20260811 22:13:50.998447 20197 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.28.47.131:37721
I20260811 22:13:51.007537 28862 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260811 22:13:51.014258 20240 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:51.009907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38766 (local address 127.28.47.190:37243)
0811 22:13:51.010085 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:51.010090 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:51.010208 (+ 118us) server_negotiation.cc:407] Connection header received
0811 22:13:51.010382 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:51.010388 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:51.010450 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:51.010544 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:51.010867 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.011509 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.012542 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.012774 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.012919 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:51.013798 (+ 879us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:51.013804 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:51.013812 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:51.013826 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:51.013846 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:51.013854 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:51.013945 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:51.014036 (+ 91us) server_negotiation.cc:299] Negotiation successful
0811 22:13:51.014102 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260811 22:13:51.017824 20197 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:38766:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260811 22:13:51.021037 20197 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260811 22:13:51.038244 20735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:51.030779 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:33693 (local address 127.0.0.1:59148)
0811 22:13:51.031655 (+ 876us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:51.031714 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:51.031856 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:51.033101 (+ 1245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:51.033108 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:51.033140 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:51.033386 (+ 246us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:51.033394 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.034890 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.034893 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:51.036360 (+ 1467us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:51.036370 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.037594 (+ 1224us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.037598 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:51.037708 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:51.037769 (+ 61us) client_negotiation.cc:769] Sending connection context
0811 22:13:51.037900 (+ 131us) client_negotiation.cc:240] Negotiation successful
0811 22:13:51.038053 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":673,"mutex_wait_us":95,"spinlock_wait_cycles":4864,"thread_start_us":163,"threads_started":1}
I20260811 22:13:51.038280 20736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:51.030934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59148 (local address 127.28.47.131:33693)
0811 22:13:51.032556 (+ 1622us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:51.032575 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:51.032602 (+ 27us) server_negotiation.cc:407] Connection header received
0811 22:13:51.032710 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:51.032715 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:51.032914 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:51.033074 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:51.033520 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.034737 (+ 1217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.036509 (+ 1772us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.037454 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.037575 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:51.037790 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:51.037945 (+ 155us) server_negotiation.cc:299] Negotiation successful
0811 22:13:51.038078 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1495,"thread_start_us":124,"threads_started":1}
I20260811 22:13:51.038508 20738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:51.031026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:39347 (local address 127.0.0.1:52800)
0811 22:13:51.031683 (+ 657us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:51.031714 (+ 31us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:51.031856 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:51.033121 (+ 1265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:51.033124 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:51.033145 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:51.033386 (+ 241us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:51.033394 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.036238 (+ 2844us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.036242 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:51.037072 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:51.037082 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.038078 (+ 996us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.038082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:51.038184 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:51.038209 (+ 25us) client_negotiation.cc:769] Sending connection context
0811 22:13:51.038279 (+ 70us) client_negotiation.cc:240] Negotiation successful
0811 22:13:51.038378 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":54,"spinlock_wait_cycles":4608,"thread_start_us":87,"threads_started":1}
I20260811 22:13:51.038619 20739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:51.031772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52800 (local address 127.28.47.130:39347)
0811 22:13:51.032405 (+ 633us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:51.032416 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:51.032440 (+ 24us) server_negotiation.cc:407] Connection header received
0811 22:13:51.032534 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:51.032539 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:51.032763 (+ 224us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:51.032944 (+ 181us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:51.034912 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.036112 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.037237 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.037949 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.038054 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:51.038271 (+ 217us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:51.038328 (+ 57us) server_negotiation.cc:299] Negotiation successful
0811 22:13:51.038450 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":507,"thread_start_us":111,"threads_started":1}
I20260811 22:13:51.039389 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:51.031239 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:39863 (local address 127.0.0.1:33280)
0811 22:13:51.032137 (+ 898us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:51.032147 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:51.032214 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:51.033869 (+ 1655us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:51.033872 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:51.033892 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:51.033972 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:51.033977 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.035273 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.035277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:51.036470 (+ 1193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:51.036485 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.039000 (+ 2515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.039003 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:51.039069 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:51.039088 (+ 19us) client_negotiation.cc:769] Sending connection context
0811 22:13:51.039189 (+ 101us) client_negotiation.cc:240] Negotiation successful
0811 22:13:51.039262 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":843,"mutex_wait_us":1,"thread_start_us":147,"threads_started":1}
I20260811 22:13:51.039832 20740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:51.031743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33280 (local address 127.28.47.129:39863)
0811 22:13:51.033389 (+ 1646us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:51.033395 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:51.033414 (+ 19us) server_negotiation.cc:407] Connection header received
0811 22:13:51.033510 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:51.033515 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:51.033707 (+ 192us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:51.033870 (+ 163us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:51.034059 (+ 189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.035134 (+ 1075us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.036621 (+ 1487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.037457 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.037575 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:51.037790 (+ 215us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:51.039238 (+ 1448us) server_negotiation.cc:299] Negotiation successful
0811 22:13:51.039350 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1539,"thread_start_us":140,"threads_started":1}
I20260811 22:13:51.041944 20496 tablet_service.cc:1467] Processing CreateTablet for tablet 2ae653a91a124b92864d166e021fa019 (DEFAULT_TABLE table=test-table [id=5ee4dcb671f44c87a561b9678081d3cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:51.041944 20647 tablet_service.cc:1467] Processing CreateTablet for tablet 2ae653a91a124b92864d166e021fa019 (DEFAULT_TABLE table=test-table [id=5ee4dcb671f44c87a561b9678081d3cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:51.043210 20496 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ae653a91a124b92864d166e021fa019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:51.043210 20647 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ae653a91a124b92864d166e021fa019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:51.045646 20338 tablet_service.cc:1467] Processing CreateTablet for tablet 2ae653a91a124b92864d166e021fa019 (DEFAULT_TABLE table=test-table [id=5ee4dcb671f44c87a561b9678081d3cc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260811 22:13:51.046720 20338 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2ae653a91a124b92864d166e021fa019. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:51.051769 20741 tablet_bootstrap.cc:492] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b: Bootstrap starting.
I20260811 22:13:51.052289 20743 tablet_bootstrap.cc:492] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb: Bootstrap starting.
I20260811 22:13:51.055450 20741 tablet_bootstrap.cc:654] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:51.055456 20743 tablet_bootstrap.cc:654] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:51.055975 20744 tablet_bootstrap.cc:492] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb: Bootstrap starting.
I20260811 22:13:51.056409 20741 log.cc:824] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:51.056452 20743 log.cc:824] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:51.058501 20744 tablet_bootstrap.cc:654] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:51.058879 20743 tablet_bootstrap.cc:492] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb: No bootstrap required, opened a new log
I20260811 22:13:51.058879 20741 tablet_bootstrap.cc:492] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b: No bootstrap required, opened a new log
I20260811 22:13:51.059110 20743 ts_tablet_manager.cc:1397] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260811 22:13:51.059110 20741 ts_tablet_manager.cc:1397] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260811 22:13:51.059531 20744 log.cc:824] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:51.061712 20744 tablet_bootstrap.cc:492] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb: No bootstrap required, opened a new log
I20260811 22:13:51.061945 20744 ts_tablet_manager.cc:1397] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260811 22:13:51.064067 20743 raft_consensus.cc:348] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffebb4067fb746998d57c421e9304e9b" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33693 } } peers { permanent_uuid: "afb8b849f1a34d7999c31ae29056addb" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39347 } } peers { permanent_uuid: "59869f03fcdc49d08b32702db054cfdb" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39863 } }
I20260811 22:13:51.064386 20743 raft_consensus.cc:374] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:51.064463 20743 raft_consensus.cc:729] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afb8b849f1a34d7999c31ae29056addb, State: Initialized, Role: FOLLOWER
I20260811 22:13:51.064996 20743 consensus_queue.cc:260] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffebb4067fb746998d57c421e9304e9b" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33693 } } peers { permanent_uuid: "afb8b849f1a34d7999c31ae29056addb" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39347 } } peers { permanent_uuid: "59869f03fcdc49d08b32702db054cfdb" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39863 } }
I20260811 22:13:51.064955 20741 raft_consensus.cc:348] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffebb4067fb746998d57c421e9304e9b" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33693 } } peers { permanent_uuid: "afb8b849f1a34d7999c31ae29056addb" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39347 } } peers { permanent_uuid: "59869f03fcdc49d08b32702db054cfdb" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39863 } }
I20260811 22:13:51.065258 20741 raft_consensus.cc:374] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:51.065322 20741 raft_consensus.cc:729] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffebb4067fb746998d57c421e9304e9b, State: Initialized, Role: FOLLOWER
I20260811 22:13:51.065653 20744 raft_consensus.cc:348] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffebb4067fb746998d57c421e9304e9b" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33693 } } peers { permanent_uuid: "afb8b849f1a34d7999c31ae29056addb" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39347 } } peers { permanent_uuid: "59869f03fcdc49d08b32702db054cfdb" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39863 } }
I20260811 22:13:51.065881 20570 heartbeater.cc:503] Master 127.28.47.190:37243 was elected leader, sending a full tablet report...
I20260811 22:13:51.065935 20744 raft_consensus.cc:374] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:51.065985 20744 raft_consensus.cc:729] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59869f03fcdc49d08b32702db054cfdb, State: Initialized, Role: FOLLOWER
I20260811 22:13:51.065822 20741 consensus_queue.cc:260] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffebb4067fb746998d57c421e9304e9b" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33693 } } peers { permanent_uuid: "afb8b849f1a34d7999c31ae29056addb" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39347 } } peers { permanent_uuid: "59869f03fcdc49d08b32702db054cfdb" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39863 } }
I20260811 22:13:51.066354 20743 ts_tablet_manager.cc:1428] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260811 22:13:51.066481 20744 consensus_queue.cc:260] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffebb4067fb746998d57c421e9304e9b" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33693 } } peers { permanent_uuid: "afb8b849f1a34d7999c31ae29056addb" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39347 } } peers { permanent_uuid: "59869f03fcdc49d08b32702db054cfdb" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39863 } }
I20260811 22:13:51.067018 20741 ts_tablet_manager.cc:1428] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
I20260811 22:13:51.067350 20723 heartbeater.cc:503] Master 127.28.47.190:37243 was elected leader, sending a full tablet report...
I20260811 22:13:51.067507 20744 ts_tablet_manager.cc:1428] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260811 22:13:51.069437 20418 heartbeater.cc:503] Master 127.28.47.190:37243 was elected leader, sending a full tablet report...
I20260811 22:13:51.143358 20751 raft_consensus.cc:482] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260811 22:13:51.143656 20751 raft_consensus.cc:504] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffebb4067fb746998d57c421e9304e9b" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33693 } } peers { permanent_uuid: "afb8b849f1a34d7999c31ae29056addb" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39347 } } peers { permanent_uuid: "59869f03fcdc49d08b32702db054cfdb" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39863 } }
I20260811 22:13:51.144970 20751 leader_election.cc:288] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ffebb4067fb746998d57c421e9304e9b (127.28.47.131:33693), afb8b849f1a34d7999c31ae29056addb (127.28.47.130:39347)
I20260811 22:13:51.149547 20736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:51.145583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:50541 (local address 127.28.47.131:33693)
0811 22:13:51.145746 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:51.145751 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:51.145867 (+ 116us) server_negotiation.cc:407] Connection header received
0811 22:13:51.146053 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:51.146061 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:51.146104 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:51.146195 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:51.146516 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.147080 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.148437 (+ 1357us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.148863 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.148995 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:51.149058 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:51.149335 (+ 277us) server_negotiation.cc:299] Negotiation successful
0811 22:13:51.149407 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260811 22:13:51.149564 20752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:51.145357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.131:33693 (local address 127.28.47.129:50541)
0811 22:13:51.145752 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:51.145769 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:51.145932 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:51.146209 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:51.146212 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:51.146229 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:51.146400 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:51.146406 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.147208 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.147211 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:51.148304 (+ 1093us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:51.148315 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.148993 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.148996 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:51.149193 (+ 197us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:51.149214 (+ 21us) client_negotiation.cc:769] Sending connection context
0811 22:13:51.149305 (+ 91us) client_negotiation.cc:240] Negotiation successful
0811 22:13:51.149395 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"mutex_wait_us":89,"thread_start_us":102,"threads_started":1}
I20260811 22:13:51.149564 20739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:51.145592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.28.47.129:41105 (local address 127.28.47.130:39347)
0811 22:13:51.145746 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:51.145752 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:51.145985 (+ 233us) server_negotiation.cc:407] Connection header received
0811 22:13:51.146143 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:51.146146 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:51.146204 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:51.146300 (+ 96us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0811 22:13:51.146629 (+ 329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.147129 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.148431 (+ 1302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.148857 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.148993 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:51.149058 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:51.149360 (+ 302us) server_negotiation.cc:299] Negotiation successful
0811 22:13:51.149439 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:13:51.149585 20754 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:51.145357 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.130:39347 (local address 127.28.47.129:41105)
0811 22:13:51.145862 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:51.145874 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0811 22:13:51.146019 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:13:51.146312 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:13:51.146315 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:13:51.146333 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0811 22:13:51.146498 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:51.146504 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.147257 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.147260 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:51.148288 (+ 1028us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:13:51.148299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.148992 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.148996 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:13:51.149225 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:51.149253 (+ 28us) client_negotiation.cc:769] Sending connection context
0811 22:13:51.149339 (+ 86us) client_negotiation.cc:240] Negotiation successful
0811 22:13:51.149419 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":111,"thread_start_us":105,"threads_started":1}
I20260811 22:13:51.150395 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ae653a91a124b92864d166e021fa019" candidate_uuid: "59869f03fcdc49d08b32702db054cfdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffebb4067fb746998d57c421e9304e9b" is_pre_election: true
I20260811 22:13:51.150429 20518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ae653a91a124b92864d166e021fa019" candidate_uuid: "59869f03fcdc49d08b32702db054cfdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afb8b849f1a34d7999c31ae29056addb" is_pre_election: true
I20260811 22:13:51.150775 20518 raft_consensus.cc:2454] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59869f03fcdc49d08b32702db054cfdb in term 0.
I20260811 22:13:51.150781 20671 raft_consensus.cc:2454] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 59869f03fcdc49d08b32702db054cfdb in term 0.
I20260811 22:13:51.151435 20285 leader_election.cc:302] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59869f03fcdc49d08b32702db054cfdb, ffebb4067fb746998d57c421e9304e9b; no voters:
I20260811 22:13:51.151813 20751 raft_consensus.cc:2781] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260811 22:13:51.151928 20751 raft_consensus.cc:482] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260811 22:13:51.151988 20751 raft_consensus.cc:3037] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:51.153591 20751 raft_consensus.cc:504] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffebb4067fb746998d57c421e9304e9b" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33693 } } peers { permanent_uuid: "afb8b849f1a34d7999c31ae29056addb" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39347 } } peers { permanent_uuid: "59869f03fcdc49d08b32702db054cfdb" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39863 } }
I20260811 22:13:51.154052 20751 leader_election.cc:288] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [CANDIDATE]: Term 1 election: Requested vote from peers ffebb4067fb746998d57c421e9304e9b (127.28.47.131:33693), afb8b849f1a34d7999c31ae29056addb (127.28.47.130:39347)
I20260811 22:13:51.154488 20518 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ae653a91a124b92864d166e021fa019" candidate_uuid: "59869f03fcdc49d08b32702db054cfdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "afb8b849f1a34d7999c31ae29056addb"
I20260811 22:13:51.154492 20671 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2ae653a91a124b92864d166e021fa019" candidate_uuid: "59869f03fcdc49d08b32702db054cfdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ffebb4067fb746998d57c421e9304e9b"
I20260811 22:13:51.154618 20518 raft_consensus.cc:3037] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:51.154652 20671 raft_consensus.cc:3037] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:51.156073 20671 raft_consensus.cc:2454] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59869f03fcdc49d08b32702db054cfdb in term 1.
I20260811 22:13:51.156093 20518 raft_consensus.cc:2454] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 59869f03fcdc49d08b32702db054cfdb in term 1.
I20260811 22:13:51.156425 20285 leader_election.cc:302] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 59869f03fcdc49d08b32702db054cfdb, ffebb4067fb746998d57c421e9304e9b; no voters:
I20260811 22:13:51.156611 20751 raft_consensus.cc:2781] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:51.156855 20751 raft_consensus.cc:686] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [term 1 LEADER]: Becoming Leader. State: Replica: 59869f03fcdc49d08b32702db054cfdb, State: Running, Role: LEADER
I20260811 22:13:51.157285 20751 consensus_queue.cc:237] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffebb4067fb746998d57c421e9304e9b" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33693 } } peers { permanent_uuid: "afb8b849f1a34d7999c31ae29056addb" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39347 } } peers { permanent_uuid: "59869f03fcdc49d08b32702db054cfdb" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39863 } }
I20260811 22:13:51.160799 20196 catalog_manager.cc:5495] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb reported cstate change: term changed from 0 to 1, leader changed from <none> to 59869f03fcdc49d08b32702db054cfdb (127.28.47.129). New cstate: current_term: 1 leader_uuid: "59869f03fcdc49d08b32702db054cfdb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ffebb4067fb746998d57c421e9304e9b" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33693 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "afb8b849f1a34d7999c31ae29056addb" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39347 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "59869f03fcdc49d08b32702db054cfdb" member_type: VOTER last_known_addr { host: "127.28.47.129" port: 39863 } health_report { overall_health: HEALTHY } } }
I20260811 22:13:51.196074 20740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:51.192076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33284 (local address 127.28.47.129:39863)
0811 22:13:51.192252 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:51.192255 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:51.192271 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:13:51.192360 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:51.192363 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:51.192424 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:51.192541 (+ 117us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:13:51.192779 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.193380 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.194374 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:51.194609 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:51.194720 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:51.195090 (+ 370us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:13:51.195570 (+ 480us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:13:51.195677 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:51.195794 (+ 117us) server_negotiation.cc:299] Negotiation successful
0811 22:13:51.195856 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":85}
I20260811 22:13:51.203007 20671 raft_consensus.cc:1261] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b [term 1 FOLLOWER]: Refusing update from remote peer 59869f03fcdc49d08b32702db054cfdb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:51.203007 20518 raft_consensus.cc:1261] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb [term 1 FOLLOWER]: Refusing update from remote peer 59869f03fcdc49d08b32702db054cfdb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:13:51.203841 20751 consensus_queue.cc:1035] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [LEADER]: Connected to new peer: Peer: permanent_uuid: "ffebb4067fb746998d57c421e9304e9b" member_type: VOTER last_known_addr { host: "127.28.47.131" port: 33693 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:51.204125 20755 consensus_queue.cc:1035] T 2ae653a91a124b92864d166e021fa019 P 59869f03fcdc49d08b32702db054cfdb [LEADER]: Connected to new peer: Peer: permanent_uuid: "afb8b849f1a34d7999c31ae29056addb" member_type: VOTER last_known_addr { host: "127.28.47.130" port: 39347 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:13:51.221558 20193 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:38766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:51.221836 20193 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:38766:
table { table_name: "test-table" } modify_external_catalogs: true
I20260811 22:13:51.224489 20193 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8baba57abc043a4928e3452d7ca5f85: Sending DeleteTablet for 3 replicas of tablet 2ae653a91a124b92864d166e021fa019
I20260811 22:13:51.225445 20647 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ae653a91a124b92864d166e021fa019 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:51 UTC) from {username='slave'} at 127.0.0.1:59148
I20260811 22:13:51.225528 20338 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ae653a91a124b92864d166e021fa019 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:51 UTC) from {username='slave'} at 127.0.0.1:33280
I20260811 22:13:51.225517 20496 tablet_service.cc:1514] Processing DeleteTablet for tablet 2ae653a91a124b92864d166e021fa019 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-11 22:13:51 UTC) from {username='slave'} at 127.0.0.1:52800
I20260811 22:13:51.226089 20769 tablet_replica.cc:330] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb: stopping tablet replica
I20260811 22:13:51.226253 20771 tablet_replica.cc:330] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b: stopping tablet replica
I20260811 22:13:51.226516 20769 raft_consensus.cc:2229] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:51.226514 20771 raft_consensus.cc:2229] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260811 22:13:51.226881 20769 raft_consensus.cc:2258] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:51.226881 20771 raft_consensus.cc:2258] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260811 22:13:51.227025 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 20264
I20260811 22:13:51.231050 20771 ts_tablet_manager.cc:1905] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:51.234403 20769 ts_tablet_manager.cc:1905] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260811 22:13:51.234661 20771 ts_tablet_manager.cc:1918] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:51.234790 20771 log.cc:1196] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-2/wal/wals/2ae653a91a124b92864d166e021fa019
I20260811 22:13:51.235139 20771 ts_tablet_manager.cc:1939] T 2ae653a91a124b92864d166e021fa019 P ffebb4067fb746998d57c421e9304e9b: Deleting consensus metadata
I20260811 22:13:51.236902 20182 catalog_manager.cc:4841] TS ffebb4067fb746998d57c421e9304e9b (127.28.47.131:33693): tablet 2ae653a91a124b92864d166e021fa019 (table test-table [id=5ee4dcb671f44c87a561b9678081d3cc]) successfully deleted
I20260811 22:13:51.237521 20769 ts_tablet_manager.cc:1918] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260811 22:13:51.237617 20769 log.cc:1196] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb: Deleting WAL directory at /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786486147647525-28862-0/minicluster-data/ts-1/wal/wals/2ae653a91a124b92864d166e021fa019
I20260811 22:13:51.237939 20769 ts_tablet_manager.cc:1939] T 2ae653a91a124b92864d166e021fa019 P afb8b849f1a34d7999c31ae29056addb: Deleting consensus metadata
I20260811 22:13:51.239177 20180 catalog_manager.cc:4841] TS afb8b849f1a34d7999c31ae29056addb (127.28.47.130:39347): tablet 2ae653a91a124b92864d166e021fa019 (table test-table [id=5ee4dcb671f44c87a561b9678081d3cc]) successfully deleted
W20260811 22:13:51.240123 20179 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.28.47.129:39863 (error 108)
I20260811 22:13:51.241003 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 20421
I20260811 22:13:51.241400 20737 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:51.240993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.129:39863 (local address 127.0.0.1:33290)
0811 22:13:51.241205 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0811 22:13:51.241293 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.129:39863: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 22:13:51.243475 20179 catalog_manager.cc:4584] TS 59869f03fcdc49d08b32702db054cfdb (127.28.47.129:39863): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2ae653a91a124b92864d166e021fa019: Network error: Client connection negotiation failed: client connection to 127.28.47.129:39863: connect: Connection refused (error 111)
I20260811 22:13:51.250590 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 20574
I20260811 22:13:51.260875 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 20160
2026-08-11T22:13:51Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1205 ms)
[----------] 6 tests from EncryptionPolicyTest (11667 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:13:51 dist-test-slave-vxj2 krb5kdc[20777](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:13:51 dist-test-slave-vxj2 krb5kdc[20777](info): set up 2 sockets
Aug 11 22:13:51 dist-test-slave-vxj2 krb5kdc[20777](info): commencing operation
krb5kdc: starting...
W20260811 22:13:52.426234 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:13:52 dist-test-slave-vxj2 krb5kdc[20777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:13:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:13:52.569988 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37731
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38611
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37731
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:52.690377 20791 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:52.690771 20791 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:52.690855 20791 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:52.695150 20791 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:52.695286 20791 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:52.695327 20791 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:52.695366 20791 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:52.695402 20791 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:52.701228 20791 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38611
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37731
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37731
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20791
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:52.702512 20791 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:52.703636 20791 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:52.713116 20802 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:52.713165 20797 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:52.713227 20791 server_base.cc:1034] running on GCE node
W20260811 22:13:52.713135 20799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:52.714087 20791 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:52.715363 20791 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:52.716547 20791 hybrid_clock.cc:648] HybridClock initialized: now 1786486432716518 us; error 44 us; skew 500 ppm
Aug 11 22:13:52 dist-test-slave-vxj2 krb5kdc[20777](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486432, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:52.719532 20791 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:13:52.720585 20791 webserver.cc:460] Webserver started at http://127.0.0.1:42811/ using document root <none> and password file <none>
I20260811 22:13:52.721143 20791 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:52.721274 20791 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:52.721474 20791 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:52.723680 20791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "6f796b5839f3491bbadf7302bccc0cc1"
format_stamp: "Formatted at 2026-08-11 22:13:52 on dist-test-slave-vxj2"
server_key: "e62eca70e40cde06396df18a822349fd"
server_key_iv: "cdfd07fa9e08c6b3273ceeff97d9c52c"
server_key_version: "encryptionkey@0"
I20260811 22:13:52.724318 20791 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "6f796b5839f3491bbadf7302bccc0cc1"
format_stamp: "Formatted at 2026-08-11 22:13:52 on dist-test-slave-vxj2"
server_key: "e62eca70e40cde06396df18a822349fd"
server_key_iv: "cdfd07fa9e08c6b3273ceeff97d9c52c"
server_key_version: "encryptionkey@0"
I20260811 22:13:52.728535 20791 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:13:52.731622 20809 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:52.733292 20791 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:52.733471 20791 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "6f796b5839f3491bbadf7302bccc0cc1"
format_stamp: "Formatted at 2026-08-11 22:13:52 on dist-test-slave-vxj2"
server_key: "e62eca70e40cde06396df18a822349fd"
server_key_iv: "cdfd07fa9e08c6b3273ceeff97d9c52c"
server_key_version: "encryptionkey@0"
I20260811 22:13:52.733613 20791 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:52.754623 20791 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:52.758379 20791 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:52.758564 20791 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:52.766337 20791 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37731
I20260811 22:13:52.766358 20872 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37731 every 8 connection(s)
I20260811 22:13:52.767429 20791 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:52.768244 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 20791
I20260811 22:13:52.768404 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:13:52.768683 28862 external_mini_cluster.cc:1428] Setting key cc04e05ace26f42c1347dba0a80963d7
I20260811 22:13:52.770861 20875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:52.778201 20875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1: Bootstrap starting.
Aug 11 22:13:52 dist-test-slave-vxj2 krb5kdc[20777](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486432, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:13:52.781234 20875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:52.782210 20875 log.cc:824] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:52.784789 20875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1: No bootstrap required, opened a new log
I20260811 22:13:52.785128 20878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:52.770379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43807 (local address 127.0.0.1:37731)
0811 22:13:52.770839 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:52.770860 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:52.770908 (+ 48us) server_negotiation.cc:407] Connection header received
0811 22:13:52.771762 (+ 854us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:52.771786 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:52.772235 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:52.772619 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:52.773456 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:52.775057 (+ 1601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:52.775975 (+ 918us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:52.776689 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:52.776859 (+ 170us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:52.779529 (+ 2670us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:52.779537 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:52.779558 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:52.779600 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:52.781717 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:52.782278 (+ 561us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:52.782285 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:52.782292 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:52.782446 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:52.782818 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:52.782821 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:52.782822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:52.783333 (+ 511us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:52.783559 (+ 226us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:52.783884 (+ 325us) server_negotiation.cc:299] Negotiation successful
0811 22:13:52.784120 (+ 236us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":307,"thread_start_us":133,"threads_started":1}
I20260811 22:13:52.788682 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:13:52.789132 20875 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f796b5839f3491bbadf7302bccc0cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37731 } }
I20260811 22:13:52.789471 20875 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:52.789559 20875 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6f796b5839f3491bbadf7302bccc0cc1, State: Initialized, Role: FOLLOWER
I20260811 22:13:52.790119 20875 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f796b5839f3491bbadf7302bccc0cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37731 } }
I20260811 22:13:52.790316 20875 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:52.790395 20875 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:52.790501 20875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:52.791915 20875 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f796b5839f3491bbadf7302bccc0cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37731 } }
I20260811 22:13:52.792311 20875 leader_election.cc:302] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6f796b5839f3491bbadf7302bccc0cc1; no voters:
I20260811 22:13:52.792724 20875 leader_election.cc:288] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:52.792855 20885 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:52.793128 20885 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [term 1 LEADER]: Becoming Leader. State: Replica: 6f796b5839f3491bbadf7302bccc0cc1, State: Running, Role: LEADER
I20260811 22:13:52.793576 20885 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f796b5839f3491bbadf7302bccc0cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37731 } }
I20260811 22:13:52.794239 20875 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:52.794917 20885 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6f796b5839f3491bbadf7302bccc0cc1. Latest consensus state: current_term: 1 leader_uuid: "6f796b5839f3491bbadf7302bccc0cc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f796b5839f3491bbadf7302bccc0cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37731 } } }
I20260811 22:13:52.795130 20885 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:52.795611 20895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:52.797194 20887 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6f796b5839f3491bbadf7302bccc0cc1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6f796b5839f3491bbadf7302bccc0cc1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37731 } } }
I20260811 22:13:52.797366 20887 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:52.800361 20895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:52.801911 20878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:52.791159 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51091 (local address 127.0.0.1:37731)
0811 22:13:52.791287 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:52.791298 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:52.791561 (+ 263us) server_negotiation.cc:407] Connection header received
0811 22:13:52.791643 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:52.791649 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:52.791718 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:52.791812 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:52.792610 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:52.793445 (+ 835us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:52.794569 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:52.794910 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:52.795024 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:52.797144 (+ 2120us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:52.797173 (+ 29us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:52.797175 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:52.797244 (+ 69us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:52.799341 (+ 2097us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:52.800008 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:52.800014 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:52.800017 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:52.800112 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:52.800554 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:52.800558 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:52.800560 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:52.800807 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:52.800962 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:52.801650 (+ 688us) server_negotiation.cc:299] Negotiation successful
0811 22:13:52.801754 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:13:52.806809 20895 catalog_manager.cc:1347] Generated new cluster ID: 10b7999290874629bff181fed87b4a7a
I20260811 22:13:52.806896 20895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:52.815371 20895 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:52.816370 20895 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:52.825917 20895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6f796b5839f3491bbadf7302bccc0cc1: Generated new TSK 0
I20260811 22:13:52.826912 20895 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:13:52.839138 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 20791
W20260811 22:13:52.848472 20775 connection.cc:535] client connection to 127.0.0.1:37731 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37731): Connection reset by peer (error 104)
2026-08-11T22:13:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1578 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:13:52 dist-test-slave-vxj2 krb5kdc[20909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:13:52 dist-test-slave-vxj2 krb5kdc[20909](info): set up 2 sockets
Aug 11 22:13:52 dist-test-slave-vxj2 krb5kdc[20909](info): commencing operation
krb5kdc: starting...
W20260811 22:13:54.014256 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:13:54 dist-test-slave-vxj2 krb5kdc[20909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486434, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:13:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:13:54.157630 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41929
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39441
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41929
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:54.283241 20926 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:54.283610 20926 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:54.283692 20926 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:54.288614 20926 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:54.288717 20926 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:54.288753 20926 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:54.288807 20926 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:54.288843 20926 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:54.294529 20926 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39441
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41929
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41929
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20926
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:54.295763 20926 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:54.296958 20926 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:54.306107 20931 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:54.306126 20932 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:54.306131 20934 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:54.306582 20926 server_base.cc:1034] running on GCE node
I20260811 22:13:54.307219 20926 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:54.308466 20926 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:54.309669 20926 hybrid_clock.cc:648] HybridClock initialized: now 1786486434309654 us; error 29 us; skew 500 ppm
Aug 11 22:13:54 dist-test-slave-vxj2 krb5kdc[20909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486434, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:54.319813 20926 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:13:54.321027 20926 webserver.cc:460] Webserver started at http://127.0.0.1:44739/ using document root <none> and password file <none>
I20260811 22:13:54.321725 20926 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:54.321834 20926 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:54.322098 20926 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:54.324347 20926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "0fa4f5ccc94e4a369f9feb61f669aaa1"
format_stamp: "Formatted at 2026-08-11 22:13:54 on dist-test-slave-vxj2"
server_key: "c96281be09b520a62016f8fab3896239"
server_key_iv: "3e5f3f22e6961b133211ef1289f5703e"
server_key_version: "encryptionkey@0"
I20260811 22:13:54.324978 20926 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "0fa4f5ccc94e4a369f9feb61f669aaa1"
format_stamp: "Formatted at 2026-08-11 22:13:54 on dist-test-slave-vxj2"
server_key: "c96281be09b520a62016f8fab3896239"
server_key_iv: "3e5f3f22e6961b133211ef1289f5703e"
server_key_version: "encryptionkey@0"
I20260811 22:13:54.329290 20926 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:13:54.332134 20942 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:54.333714 20926 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:54.333917 20926 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "0fa4f5ccc94e4a369f9feb61f669aaa1"
format_stamp: "Formatted at 2026-08-11 22:13:54 on dist-test-slave-vxj2"
server_key: "c96281be09b520a62016f8fab3896239"
server_key_iv: "3e5f3f22e6961b133211ef1289f5703e"
server_key_version: "encryptionkey@0"
I20260811 22:13:54.334087 20926 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:54.388674 20926 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:54.392599 20926 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:54.392863 20926 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:54.401036 20926 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41929
I20260811 22:13:54.401069 21002 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41929 every 8 connection(s)
I20260811 22:13:54.402139 20926 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:54.405478 21003 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:54.407352 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 20926
I20260811 22:13:54.407486 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:13:54.407810 28862 external_mini_cluster.cc:1428] Setting key e348ab94239f0a8c0a3cd2d099a34813
I20260811 22:13:54.413439 21003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1: Bootstrap starting.
I20260811 22:13:54.416574 21003 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:54.417667 21003 log.cc:824] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:54.419047 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:54.409484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53781 (local address 127.0.0.1:41929)
0811 22:13:54.409999 (+ 515us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:54.410022 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:54.410072 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:13:54.410982 (+ 910us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:54.411009 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:54.411473 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:54.411886 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:54.412383 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:54.413866 (+ 1483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:54.414798 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:54.415513 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:54.415725 (+ 212us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:54.416790 (+ 1065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:54.416798 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:54.416815 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:54.417430 (+ 615us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:54.417513 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:54.417553 (+ 40us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:54.417728 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:54.417933 (+ 205us) server_negotiation.cc:299] Negotiation successful
0811 22:13:54.418088 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342,"thread_start_us":142,"threads_started":1}
I20260811 22:13:54.420159 21003 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1: No bootstrap required, opened a new log
I20260811 22:13:54.422402 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:13:54.424649 21003 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa4f5ccc94e4a369f9feb61f669aaa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41929 } }
I20260811 22:13:54.425014 21003 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:54.425091 21003 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0fa4f5ccc94e4a369f9feb61f669aaa1, State: Initialized, Role: FOLLOWER
I20260811 22:13:54.425617 21003 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa4f5ccc94e4a369f9feb61f669aaa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41929 } }
I20260811 22:13:54.425797 21003 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:54.425899 21003 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:54.426040 21003 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:54.427177 21003 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa4f5ccc94e4a369f9feb61f669aaa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41929 } }
I20260811 22:13:54.427583 21003 leader_election.cc:302] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0fa4f5ccc94e4a369f9feb61f669aaa1; no voters:
I20260811 22:13:54.427992 21003 leader_election.cc:288] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:54.428063 21016 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:54.428401 21016 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [term 1 LEADER]: Becoming Leader. State: Replica: 0fa4f5ccc94e4a369f9feb61f669aaa1, State: Running, Role: LEADER
I20260811 22:13:54.428844 21016 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa4f5ccc94e4a369f9feb61f669aaa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41929 } }
I20260811 22:13:54.429339 21007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:54.424577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50349 (local address 127.0.0.1:41929)
0811 22:13:54.424706 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:54.424713 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:54.424889 (+ 176us) server_negotiation.cc:407] Connection header received
0811 22:13:54.425061 (+ 172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:54.425064 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:54.425141 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:54.425237 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:54.425574 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:54.426325 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:54.427158 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:54.427476 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:54.427590 (+ 114us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:54.428585 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:54.428588 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:54.428591 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:54.428604 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:54.428635 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:54.428649 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:54.428737 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:54.429115 (+ 378us) server_negotiation.cc:299] Negotiation successful
0811 22:13:54.429187 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:13:54.429752 21003 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:54.431113 21017 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0fa4f5ccc94e4a369f9feb61f669aaa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa4f5ccc94e4a369f9feb61f669aaa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41929 } } }
I20260811 22:13:54.431248 21017 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:54.431190 21018 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0fa4f5ccc94e4a369f9feb61f669aaa1. Latest consensus state: current_term: 1 leader_uuid: "0fa4f5ccc94e4a369f9feb61f669aaa1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0fa4f5ccc94e4a369f9feb61f669aaa1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41929 } } }
I20260811 22:13:54.431314 21018 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:54.431722 21025 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:54.435590 21025 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:54.442027 21025 catalog_manager.cc:1347] Generated new cluster ID: ce8923b56ed04144a8825c5045de4288
I20260811 22:13:54.442137 21025 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:54.458408 21025 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:54.459826 21025 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:54.479470 21025 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0fa4f5ccc94e4a369f9feb61f669aaa1: Generated new TSK 0
I20260811 22:13:54.480583 21025 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:13:54.485880 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 20926
W20260811 22:13:54.497366 20907 connection.cc:535] client connection to 127.0.0.1:41929 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41929): Connection reset by peer (error 104)
2026-08-11T22:13:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1649 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:13:54 dist-test-slave-vxj2 krb5kdc[21040](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:13:54 dist-test-slave-vxj2 krb5kdc[21040](info): set up 2 sockets
Aug 11 22:13:54 dist-test-slave-vxj2 krb5kdc[21040](info): commencing operation
krb5kdc: starting...
W20260811 22:13:55.641474 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:13:55 dist-test-slave-vxj2 krb5kdc[21040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486435, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:13:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:13:55.777402 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40477
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36683
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40477
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:55.898816 21055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:55.899138 21055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:55.899183 21055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:55.903568 21055 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:55.903667 21055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:55.903687 21055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:55.903704 21055 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:55.903721 21055 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:55.909701 21055 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36683
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40477
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40477
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:55.911002 21055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:55.912150 21055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:55.921867 21063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:55.921852 21060 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:55.922004 21055 server_base.cc:1034] running on GCE node
W20260811 22:13:55.921851 21061 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:55.922737 21055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:55.923990 21055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:55.925241 21055 hybrid_clock.cc:648] HybridClock initialized: now 1786486435925203 us; error 42 us; skew 500 ppm
Aug 11 22:13:55 dist-test-slave-vxj2 krb5kdc[21040](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486435, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:55.928310 21055 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:13:55.929363 21055 webserver.cc:460] Webserver started at http://127.0.0.1:37333/ using document root <none> and password file <none>
I20260811 22:13:55.929966 21055 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:55.930063 21055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:55.930307 21055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:55.932453 21055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "86c6947df6b54254b6fa3a71df3f551f"
format_stamp: "Formatted at 2026-08-11 22:13:55 on dist-test-slave-vxj2"
server_key: "06cb0b11885398f2fff65e184faa0923"
server_key_iv: "0db7970f8f6a97d34939e6149ffc6d9a"
server_key_version: "encryptionkey@0"
I20260811 22:13:55.933058 21055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "86c6947df6b54254b6fa3a71df3f551f"
format_stamp: "Formatted at 2026-08-11 22:13:55 on dist-test-slave-vxj2"
server_key: "06cb0b11885398f2fff65e184faa0923"
server_key_iv: "0db7970f8f6a97d34939e6149ffc6d9a"
server_key_version: "encryptionkey@0"
I20260811 22:13:55.937383 21055 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:13:55.940148 21070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:55.941874 21055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:55.942057 21055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "86c6947df6b54254b6fa3a71df3f551f"
format_stamp: "Formatted at 2026-08-11 22:13:55 on dist-test-slave-vxj2"
server_key: "06cb0b11885398f2fff65e184faa0923"
server_key_iv: "0db7970f8f6a97d34939e6149ffc6d9a"
server_key_version: "encryptionkey@0"
I20260811 22:13:55.942206 21055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:55.947562 21055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:55.951362 21055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:55.951596 21055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:55.959719 21055 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40477
I20260811 22:13:55.959740 21126 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40477 every 8 connection(s)
I20260811 22:13:55.960915 21055 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:55.964093 21127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:55.965559 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 21055
I20260811 22:13:55.965726 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:13:55.966025 28862 external_mini_cluster.cc:1428] Setting key 2ce1213ba279b2d8d5dc743265802309
I20260811 22:13:55.971274 21127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f: Bootstrap starting.
I20260811 22:13:55.972167 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:55.967634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52689 (local address 127.0.0.1:40477)
0811 22:13:55.968094 (+ 460us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:55.968110 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:55.968148 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:13:55.968841 (+ 693us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:55.968859 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:55.969212 (+ 353us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:55.969524 (+ 312us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:55.970499 (+ 975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:55.970505 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:55.970541 (+ 36us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:55.970556 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:55.970653 (+ 97us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:55.970680 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:55.970841 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:55.970961 (+ 120us) server_negotiation.cc:299] Negotiation successful
0811 22:13:55.971182 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":150,"threads_started":1}
I20260811 22:13:55.974112 21127 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:55.975075 21127 log.cc:824] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:55.975668 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:13:55.977505 21127 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f: No bootstrap required, opened a new log
I20260811 22:13:55.979636 21132 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:55.977941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48589 (local address 127.0.0.1:40477)
0811 22:13:55.978081 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:55.978086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:55.978264 (+ 178us) server_negotiation.cc:407] Connection header received
0811 22:13:55.978371 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:55.978374 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:55.978414 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:55.978497 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:55.979246 (+ 749us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:55.979250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:55.979252 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:55.979265 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:55.979298 (+ 33us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:55.979312 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:55.979404 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:55.979466 (+ 62us) server_negotiation.cc:299] Negotiation successful
0811 22:13:55.979508 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260811 22:13:55.982383 21127 raft_consensus.cc:348] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c6947df6b54254b6fa3a71df3f551f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40477 } }
I20260811 22:13:55.982683 21127 raft_consensus.cc:374] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:55.982749 21127 raft_consensus.cc:729] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 86c6947df6b54254b6fa3a71df3f551f, State: Initialized, Role: FOLLOWER
I20260811 22:13:55.983199 21127 consensus_queue.cc:260] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c6947df6b54254b6fa3a71df3f551f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40477 } }
I20260811 22:13:55.983367 21127 raft_consensus.cc:388] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:55.983461 21127 raft_consensus.cc:482] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:55.983577 21127 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:55.984681 21127 raft_consensus.cc:504] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c6947df6b54254b6fa3a71df3f551f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40477 } }
I20260811 22:13:55.985095 21127 leader_election.cc:302] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 86c6947df6b54254b6fa3a71df3f551f; no voters:
I20260811 22:13:55.985556 21127 leader_election.cc:288] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:55.985800 21141 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:55.986090 21141 raft_consensus.cc:686] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [term 1 LEADER]: Becoming Leader. State: Replica: 86c6947df6b54254b6fa3a71df3f551f, State: Running, Role: LEADER
I20260811 22:13:55.986470 21141 consensus_queue.cc:237] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c6947df6b54254b6fa3a71df3f551f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40477 } }
I20260811 22:13:55.986971 21127 sys_catalog.cc:564] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:55.988440 21142 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "86c6947df6b54254b6fa3a71df3f551f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c6947df6b54254b6fa3a71df3f551f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40477 } } }
I20260811 22:13:55.988633 21142 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:55.989009 21144 sys_catalog.cc:455] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 86c6947df6b54254b6fa3a71df3f551f. Latest consensus state: current_term: 1 leader_uuid: "86c6947df6b54254b6fa3a71df3f551f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "86c6947df6b54254b6fa3a71df3f551f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40477 } } }
I20260811 22:13:55.989105 21144 sys_catalog.cc:458] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f [sys.catalog]: This master's current role is: LEADER
W20260811 22:13:55.991806 21160 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:13:55.991881 21160 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:13:55.991969 21155 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:55.992642 21155 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:55.999037 21155 catalog_manager.cc:1347] Generated new cluster ID: 835360b0fecd448482084ef1ce36d497
I20260811 22:13:55.999159 21155 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:56.028057 21155 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:56.028465 21155 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:56.039850 21155 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 86c6947df6b54254b6fa3a71df3f551f: Generated new TSK 0
I20260811 22:13:56.040781 21155 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:13:56.051874 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 21055
2026-08-11T22:13:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1564 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:13:56 dist-test-slave-vxj2 krb5kdc[21166](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:13:56 dist-test-slave-vxj2 krb5kdc[21166](info): set up 2 sockets
Aug 11 22:13:56 dist-test-slave-vxj2 krb5kdc[21166](info): commencing operation
krb5kdc: starting...
W20260811 22:13:57.217022 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:13:57 dist-test-slave-vxj2 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:13:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:13:57.356926 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34753
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34753
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:57.483541 21181 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:57.483934 21181 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:57.484030 21181 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:57.488412 21181 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:57.488546 21181 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:57.488582 21181 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:57.488622 21181 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:57.488682 21181 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:57.494474 21181 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34753
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34753
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21181
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:57.495749 21181 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:57.496874 21181 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:57.506312 21187 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:57.506494 21181 server_base.cc:1034] running on GCE node
W20260811 22:13:57.506347 21189 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:57.506707 21191 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:57.507323 21181 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:57.508605 21181 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:57.509804 21181 hybrid_clock.cc:648] HybridClock initialized: now 1786486437509784 us; error 36 us; skew 500 ppm
Aug 11 22:13:57 dist-test-slave-vxj2 krb5kdc[21166](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486437, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:57.512749 21181 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:13:57.513818 21181 webserver.cc:460] Webserver started at http://127.0.0.1:43821/ using document root <none> and password file <none>
I20260811 22:13:57.514379 21181 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:57.514451 21181 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:57.514635 21181 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:57.516968 21181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "4ea89ccb4aa7415b93d8de2474f0b248"
format_stamp: "Formatted at 2026-08-11 22:13:57 on dist-test-slave-vxj2"
server_key: "9e17e28afdc8439d4fab381f5afffa70"
server_key_iv: "8812e68d1d7b889681e45b18bd13c790"
server_key_version: "encryptionkey@0"
I20260811 22:13:57.517695 21181 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "4ea89ccb4aa7415b93d8de2474f0b248"
format_stamp: "Formatted at 2026-08-11 22:13:57 on dist-test-slave-vxj2"
server_key: "9e17e28afdc8439d4fab381f5afffa70"
server_key_iv: "8812e68d1d7b889681e45b18bd13c790"
server_key_version: "encryptionkey@0"
I20260811 22:13:57.522445 21181 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:13:57.525283 21201 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:57.526769 21181 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:13:57.526968 21181 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "4ea89ccb4aa7415b93d8de2474f0b248"
format_stamp: "Formatted at 2026-08-11 22:13:57 on dist-test-slave-vxj2"
server_key: "9e17e28afdc8439d4fab381f5afffa70"
server_key_iv: "8812e68d1d7b889681e45b18bd13c790"
server_key_version: "encryptionkey@0"
I20260811 22:13:57.527118 21181 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:57.544857 21181 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:57.548280 21181 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:57.548532 21181 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:57.556277 21181 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34753
I20260811 22:13:57.556296 21263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34753 every 8 connection(s)
I20260811 22:13:57.557437 21181 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:57.560456 21264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:57.565848 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 21181
I20260811 22:13:57.566013 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:13:57.566309 28862 external_mini_cluster.cc:1428] Setting key b43dc8a0d7e269b76581123570d5d05a
I20260811 22:13:57.567351 21264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248: Bootstrap starting.
I20260811 22:13:57.569970 21264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:57.571058 21264 log.cc:824] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:57.573581 21264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248: No bootstrap required, opened a new log
Aug 11 22:13:57 dist-test-slave-vxj2 krb5kdc[21166](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486437, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:13:57.578624 21264 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea89ccb4aa7415b93d8de2474f0b248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34753 } }
I20260811 22:13:57.578934 21264 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:57.579026 21264 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4ea89ccb4aa7415b93d8de2474f0b248, State: Initialized, Role: FOLLOWER
I20260811 22:13:57.579578 21264 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea89ccb4aa7415b93d8de2474f0b248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34753 } }
I20260811 22:13:57.579747 21264 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:57.579824 21264 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:57.579985 21264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:57.581049 21264 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea89ccb4aa7415b93d8de2474f0b248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34753 } }
I20260811 22:13:57.581476 21264 leader_election.cc:302] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4ea89ccb4aa7415b93d8de2474f0b248; no voters:
I20260811 22:13:57.581868 21264 leader_election.cc:288] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:57.581990 21269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:57.582216 21269 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [term 1 LEADER]: Becoming Leader. State: Replica: 4ea89ccb4aa7415b93d8de2474f0b248, State: Running, Role: LEADER
I20260811 22:13:57.583033 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:57.568023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60035 (local address 127.0.0.1:34753)
0811 22:13:57.568551 (+ 528us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:57.568570 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:57.568611 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:13:57.569437 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:57.569459 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:57.569890 (+ 431us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:57.570266 (+ 376us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:57.571288 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:57.572384 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:57.573629 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:57.574190 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:57.574350 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:57.577256 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:57.577261 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:57.577278 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:57.577308 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:57.579248 (+ 1940us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:57.579750 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:57.579756 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:57.579763 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:57.580011 (+ 248us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:57.580315 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:57.580319 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:57.580321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:57.580812 (+ 491us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:57.581033 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:57.581399 (+ 366us) server_negotiation.cc:299] Negotiation successful
0811 22:13:57.581769 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":178,"threads_started":1}
I20260811 22:13:57.583420 21264 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:57.583398 21269 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea89ccb4aa7415b93d8de2474f0b248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34753 } }
I20260811 22:13:57.585388 21271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4ea89ccb4aa7415b93d8de2474f0b248. Latest consensus state: current_term: 1 leader_uuid: "4ea89ccb4aa7415b93d8de2474f0b248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea89ccb4aa7415b93d8de2474f0b248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34753 } } }
I20260811 22:13:57.585456 21270 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4ea89ccb4aa7415b93d8de2474f0b248" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4ea89ccb4aa7415b93d8de2474f0b248" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34753 } } }
I20260811 22:13:57.585531 21271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:57.585567 21270 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:57.586058 21284 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:57.586825 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:13:57.589749 21284 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:57.596513 21284 catalog_manager.cc:1347] Generated new cluster ID: 4ae23f5e76c84f3b824d11866356dbc5
I20260811 22:13:57.596678 21284 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:57.598269 21267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:57.589075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39751 (local address 127.0.0.1:34753)
0811 22:13:57.589220 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:57.589225 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:57.589464 (+ 239us) server_negotiation.cc:407] Connection header received
0811 22:13:57.589607 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:57.589610 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:57.589664 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:57.589751 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:57.590483 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:57.591011 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:57.591843 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:57.592180 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:57.592298 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:57.594381 (+ 2083us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:57.594385 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:57.594388 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:13:57.594419 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:57.596228 (+ 1809us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:57.596810 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:57.596813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:57.596816 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:57.596916 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:13:57.597277 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:13:57.597280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:13:57.597282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:13:57.597537 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:57.597675 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:57.597980 (+ 305us) server_negotiation.cc:299] Negotiation successful
0811 22:13:57.598109 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:13:57.620954 21284 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:57.622447 21284 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:57.638082 21284 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4ea89ccb4aa7415b93d8de2474f0b248: Generated new TSK 0
I20260811 22:13:57.639156 21284 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:13:57.653546 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 21181
2026-08-11T22:13:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1599 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:13:57 dist-test-slave-vxj2 krb5kdc[21299](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:13:57 dist-test-slave-vxj2 krb5kdc[21299](info): set up 2 sockets
Aug 11 22:13:57 dist-test-slave-vxj2 krb5kdc[21299](info): commencing operation
krb5kdc: starting...
W20260811 22:13:58.797453 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.121s user 0.005s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:13:58 dist-test-slave-vxj2 krb5kdc[21299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486438, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:13:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:13:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:13:58.937095 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41471
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41471
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:13:59.063555 21315 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:13:59.063968 21315 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:13:59.064070 21315 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:13:59.068405 21315 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:13:59.068534 21315 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:13:59.068572 21315 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:13:59.068610 21315 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:13:59.068648 21315 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:13:59.074549 21315 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41471
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41471
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21315
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:13:59.075852 21315 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:13:59.077091 21315 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:13:59.087101 21323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:59.087101 21325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:13:59.087114 21322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:13:59.087695 21315 server_base.cc:1034] running on GCE node
I20260811 22:13:59.088366 21315 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:13:59.089816 21315 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:13:59.091028 21315 hybrid_clock.cc:648] HybridClock initialized: now 1786486439090998 us; error 49 us; skew 500 ppm
Aug 11 22:13:59 dist-test-slave-vxj2 krb5kdc[21299](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486439, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:13:59.094188 21315 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:13:59.095281 21315 webserver.cc:460] Webserver started at http://127.0.0.1:34023/ using document root <none> and password file <none>
I20260811 22:13:59.095889 21315 fs_manager.cc:359] Metadata directory not provided
I20260811 22:13:59.095988 21315 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:13:59.096237 21315 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:13:59.098572 21315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "9e45170ae0034d3cb885edaa27952fe9"
format_stamp: "Formatted at 2026-08-11 22:13:59 on dist-test-slave-vxj2"
server_key: "1a14512db2135cc765770eb6ecd141f3"
server_key_iv: "3b321c6ece9c66194e709cf5148b5fa8"
server_key_version: "encryptionkey@0"
I20260811 22:13:59.099205 21315 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "9e45170ae0034d3cb885edaa27952fe9"
format_stamp: "Formatted at 2026-08-11 22:13:59 on dist-test-slave-vxj2"
server_key: "1a14512db2135cc765770eb6ecd141f3"
server_key_iv: "3b321c6ece9c66194e709cf5148b5fa8"
server_key_version: "encryptionkey@0"
I20260811 22:13:59.103299 21315 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:13:59.106040 21334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:13:59.107498 21315 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:13:59.107687 21315 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "9e45170ae0034d3cb885edaa27952fe9"
format_stamp: "Formatted at 2026-08-11 22:13:59 on dist-test-slave-vxj2"
server_key: "1a14512db2135cc765770eb6ecd141f3"
server_key_iv: "3b321c6ece9c66194e709cf5148b5fa8"
server_key_version: "encryptionkey@0"
I20260811 22:13:59.107831 21315 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:13:59.135325 21315 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:13:59.139081 21315 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:13:59.139317 21315 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:13:59.150445 21315 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41471
I20260811 22:13:59.150449 21393 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41471 every 8 connection(s)
I20260811 22:13:59.152158 21315 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:13:59.155769 21394 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:13:59.156288 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 21315
I20260811 22:13:59.156481 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:13:59.156785 28862 external_mini_cluster.cc:1428] Setting key 303e7b07983976ed4f5d249cc6fb6bd9
I20260811 22:13:59.163542 21394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9: Bootstrap starting.
I20260811 22:13:59.166239 21394 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9: Neither blocks nor log segments found. Creating new log.
I20260811 22:13:59.166730 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:59.158325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41099 (local address 127.0.0.1:41471)
0811 22:13:59.158875 (+ 550us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:59.158891 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:59.158935 (+ 44us) server_negotiation.cc:407] Connection header received
0811 22:13:59.159672 (+ 737us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:59.159695 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:59.160094 (+ 399us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:59.160502 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:59.160980 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:59.162126 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:59.163116 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:59.163645 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:59.163832 (+ 187us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:59.164771 (+ 939us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:59.164776 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:59.164790 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:59.165270 (+ 480us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:59.165331 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:59.165356 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:59.165466 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:59.165644 (+ 178us) server_negotiation.cc:299] Negotiation successful
0811 22:13:59.165884 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":383,"thread_start_us":162,"threads_started":1}
I20260811 22:13:59.167217 21394 log.cc:824] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:13:59.169739 21394 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9: No bootstrap required, opened a new log
I20260811 22:13:59.169994 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:13:59.174216 21394 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e45170ae0034d3cb885edaa27952fe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41471 } }
I20260811 22:13:59.174537 21394 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:13:59.174589 21394 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e45170ae0034d3cb885edaa27952fe9, State: Initialized, Role: FOLLOWER
I20260811 22:13:59.175076 21394 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e45170ae0034d3cb885edaa27952fe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41471 } }
I20260811 22:13:59.175310 21394 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:13:59.175401 21394 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:13:59.175527 21394 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:13:59.176539 21397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:13:59.172082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57577 (local address 127.0.0.1:41471)
0811 22:13:59.172234 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0811 22:13:59.172238 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:13:59.172545 (+ 307us) server_negotiation.cc:407] Connection header received
0811 22:13:59.172685 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:13:59.172688 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:13:59.172731 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:13:59.172799 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:13:59.173122 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:59.173692 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:59.174777 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:13:59.175083 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:13:59.175166 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:13:59.176092 (+ 926us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:13:59.176095 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:13:59.176098 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:13:59.176117 (+ 19us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:13:59.176144 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:13:59.176154 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:13:59.176212 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0811 22:13:59.176327 (+ 115us) server_negotiation.cc:299] Negotiation successful
0811 22:13:59.176374 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260811 22:13:59.176940 21394 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e45170ae0034d3cb885edaa27952fe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41471 } }
I20260811 22:13:59.177372 21394 leader_election.cc:302] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e45170ae0034d3cb885edaa27952fe9; no voters:
I20260811 22:13:59.177767 21394 leader_election.cc:288] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:13:59.177887 21406 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:13:59.178079 21406 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [term 1 LEADER]: Becoming Leader. State: Replica: 9e45170ae0034d3cb885edaa27952fe9, State: Running, Role: LEADER
I20260811 22:13:59.178489 21406 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e45170ae0034d3cb885edaa27952fe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41471 } }
I20260811 22:13:59.179272 21394 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:13:59.180433 21407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e45170ae0034d3cb885edaa27952fe9. Latest consensus state: current_term: 1 leader_uuid: "9e45170ae0034d3cb885edaa27952fe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e45170ae0034d3cb885edaa27952fe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41471 } } }
I20260811 22:13:59.180585 21407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:59.180899 21408 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e45170ae0034d3cb885edaa27952fe9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e45170ae0034d3cb885edaa27952fe9" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41471 } } }
I20260811 22:13:59.181056 21408 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:13:59.181034 21416 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:13:59.184921 21416 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:13:59.191020 21416 catalog_manager.cc:1347] Generated new cluster ID: 47837e4f0f7544cd9f29121e8b8dfe78
I20260811 22:13:59.191123 21416 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:13:59.204491 21416 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:13:59.205930 21416 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:13:59.214532 21416 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e45170ae0034d3cb885edaa27952fe9: Generated new TSK 0
I20260811 22:13:59.215461 21416 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:13:59.220062 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 21315
2026-08-11T22:13:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1568 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:13:59 dist-test-slave-vxj2 krb5kdc[21429](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:13:59 dist-test-slave-vxj2 krb5kdc[21429](info): set up 2 sockets
Aug 11 22:13:59 dist-test-slave-vxj2 krb5kdc[21429](info): commencing operation
krb5kdc: starting...
W20260811 22:14:00.398864 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.154s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:00 dist-test-slave-vxj2 krb5kdc[21429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486440, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:14:00.541455 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39911
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41233
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:00.663112 21445 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:00.663466 21445 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:00.663516 21445 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:00.667929 21445 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:00.668035 21445 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:00.668061 21445 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:00.668082 21445 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:00.668138 21445 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:00.674252 21445 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41233
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39911
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21445
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:00.675612 21445 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:00.676803 21445 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:00.686141 21454 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:00.686154 21452 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:00.686305 21445 server_base.cc:1034] running on GCE node
W20260811 22:14:00.686581 21451 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:00.687317 21445 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:00.688580 21445 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:00.689821 21445 hybrid_clock.cc:648] HybridClock initialized: now 1786486440689800 us; error 34 us; skew 500 ppm
Aug 11 22:14:00 dist-test-slave-vxj2 krb5kdc[21429](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486440, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:00.697491 21445 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:14:00.698746 21445 webserver.cc:460] Webserver started at http://127.0.0.1:39379/ using document root <none> and password file <none>
I20260811 22:14:00.699407 21445 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:00.699514 21445 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:00.699784 21445 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:00.702216 21445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "9e522f8a309f40c3bbdd6c133007d51a"
format_stamp: "Formatted at 2026-08-11 22:14:00 on dist-test-slave-vxj2"
server_key: "e74c9e8c90228e80c083de5c555dee67"
server_key_iv: "090a593161da5ee3a5ea53af948f6ff5"
server_key_version: "encryptionkey@0"
I20260811 22:14:00.702904 21445 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "9e522f8a309f40c3bbdd6c133007d51a"
format_stamp: "Formatted at 2026-08-11 22:14:00 on dist-test-slave-vxj2"
server_key: "e74c9e8c90228e80c083de5c555dee67"
server_key_iv: "090a593161da5ee3a5ea53af948f6ff5"
server_key_version: "encryptionkey@0"
I20260811 22:14:00.707297 21445 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:14:00.710336 21464 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:00.711905 21445 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:14:00.712102 21445 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "9e522f8a309f40c3bbdd6c133007d51a"
format_stamp: "Formatted at 2026-08-11 22:14:00 on dist-test-slave-vxj2"
server_key: "e74c9e8c90228e80c083de5c555dee67"
server_key_iv: "090a593161da5ee3a5ea53af948f6ff5"
server_key_version: "encryptionkey@0"
I20260811 22:14:00.712249 21445 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:00.717828 21445 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:00.721457 21445 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:00.721693 21445 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:00.729755 21445 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39911
I20260811 22:14:00.729862 21522 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39911 every 8 connection(s)
I20260811 22:14:00.730898 21445 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:00.734141 21523 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:00.739778 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 21445
I20260811 22:14:00.739954 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:00.740222 28862 external_mini_cluster.cc:1428] Setting key cd66b4a6ba08a4aaeaa9f4767f77c44d
I20260811 22:14:00.740365 21523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a: Bootstrap starting.
I20260811 22:14:00.743090 21523 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:00.744273 21523 log.cc:824] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:00.745965 21527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:00.741664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49535 (local address 127.0.0.1:39911)
0811 22:14:00.742125 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:00.742141 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:00.742177 (+ 36us) server_negotiation.cc:407] Connection header received
0811 22:14:00.742839 (+ 662us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:00.742858 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:00.743160 (+ 302us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:00.743523 (+ 363us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:00.744558 (+ 1035us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:00.744563 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:00.744576 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:00.744587 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:00.744648 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:00.744668 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:00.744779 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:00.744987 (+ 208us) server_negotiation.cc:299] Negotiation successful
0811 22:14:00.745208 (+ 221us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":147,"threads_started":1}
I20260811 22:14:00.746733 21523 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a: No bootstrap required, opened a new log
I20260811 22:14:00.748927 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:00.751821 21523 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e522f8a309f40c3bbdd6c133007d51a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39911 } }
I20260811 22:14:00.752192 21523 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:00.752301 21523 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9e522f8a309f40c3bbdd6c133007d51a, State: Initialized, Role: FOLLOWER
I20260811 22:14:00.752875 21523 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e522f8a309f40c3bbdd6c133007d51a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39911 } }
I20260811 22:14:00.753062 21523 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:00.753253 21523 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:00.753391 21523 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:00.753509 21527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:00.751564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41369 (local address 127.0.0.1:39911)
0811 22:14:00.751708 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:00.751713 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:00.751872 (+ 159us) server_negotiation.cc:407] Connection header received
0811 22:14:00.751958 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:00.751961 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:00.752001 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:00.752094 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:00.752961 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:00.752965 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:00.752967 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:00.752979 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:00.753011 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:00.753046 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:00.753146 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:00.753248 (+ 102us) server_negotiation.cc:299] Negotiation successful
0811 22:14:00.753329 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:14:00.754614 21523 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e522f8a309f40c3bbdd6c133007d51a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39911 } }
I20260811 22:14:00.755002 21523 leader_election.cc:302] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9e522f8a309f40c3bbdd6c133007d51a; no voters:
I20260811 22:14:00.755371 21523 leader_election.cc:288] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:00.755542 21535 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:00.755879 21535 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [term 1 LEADER]: Becoming Leader. State: Replica: 9e522f8a309f40c3bbdd6c133007d51a, State: Running, Role: LEADER
I20260811 22:14:00.756202 21535 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e522f8a309f40c3bbdd6c133007d51a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39911 } }
I20260811 22:14:00.756803 21523 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:00.757992 21540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9e522f8a309f40c3bbdd6c133007d51a. Latest consensus state: current_term: 1 leader_uuid: "9e522f8a309f40c3bbdd6c133007d51a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e522f8a309f40c3bbdd6c133007d51a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39911 } } }
I20260811 22:14:00.758128 21540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:00.758540 21548 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:00.758500 21539 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9e522f8a309f40c3bbdd6c133007d51a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9e522f8a309f40c3bbdd6c133007d51a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39911 } } }
I20260811 22:14:00.759366 21539 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:00.762068 21548 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:00.769335 21548 catalog_manager.cc:1347] Generated new cluster ID: ef9444188647495fabd6aa02a42d8e03
I20260811 22:14:00.769459 21548 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:00.784569 21548 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:00.784982 21548 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:00.795677 21548 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9e522f8a309f40c3bbdd6c133007d51a: Generated new TSK 0
I20260811 22:14:00.796574 21548 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:00.804674 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 21445
2026-08-11T22:14:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1585 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:00 dist-test-slave-vxj2 krb5kdc[21560](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:00 dist-test-slave-vxj2 krb5kdc[21560](info): set up 2 sockets
Aug 11 22:14:00 dist-test-slave-vxj2 krb5kdc[21560](info): commencing operation
krb5kdc: starting...
W20260811 22:14:01.956476 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:02 dist-test-slave-vxj2 krb5kdc[21560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:14:02.094003 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40339
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35109
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40339
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:02.213635 21575 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:02.213979 21575 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:02.214058 21575 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:02.218451 21575 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:02.218565 21575 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:02.218606 21575 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:02.218647 21575 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:02.218680 21575 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:02.224316 21575 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35109
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40339
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40339
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21575
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:02.225642 21575 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:02.226812 21575 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:02.236307 21583 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:02.236307 21586 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:02.236462 21582 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:02.237171 21575 server_base.cc:1034] running on GCE node
I20260811 22:14:02.237900 21575 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:02.239202 21575 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:02.240435 21575 hybrid_clock.cc:648] HybridClock initialized: now 1786486442240417 us; error 36 us; skew 500 ppm
Aug 11 22:14:02 dist-test-slave-vxj2 krb5kdc[21560](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486442, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:02.243623 21575 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:14:02.244771 21575 webserver.cc:460] Webserver started at http://127.0.0.1:33185/ using document root <none> and password file <none>
I20260811 22:14:02.245483 21575 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:02.245586 21575 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:02.245859 21575 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:02.248171 21575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "070f22a756ea42f6a69d9243eca694a7"
format_stamp: "Formatted at 2026-08-11 22:14:02 on dist-test-slave-vxj2"
server_key: "f7a2f7d99d5c2d4539b807ee0c29ee89"
server_key_iv: "9b8f7878aef8013fb1b0b18bf1d1c965"
server_key_version: "encryptionkey@0"
I20260811 22:14:02.248831 21575 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "070f22a756ea42f6a69d9243eca694a7"
format_stamp: "Formatted at 2026-08-11 22:14:02 on dist-test-slave-vxj2"
server_key: "f7a2f7d99d5c2d4539b807ee0c29ee89"
server_key_iv: "9b8f7878aef8013fb1b0b18bf1d1c965"
server_key_version: "encryptionkey@0"
I20260811 22:14:02.252970 21575 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260811 22:14:02.255653 21594 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:02.256951 21575 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260811 22:14:02.257110 21575 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "070f22a756ea42f6a69d9243eca694a7"
format_stamp: "Formatted at 2026-08-11 22:14:02 on dist-test-slave-vxj2"
server_key: "f7a2f7d99d5c2d4539b807ee0c29ee89"
server_key_iv: "9b8f7878aef8013fb1b0b18bf1d1c965"
server_key_version: "encryptionkey@0"
I20260811 22:14:02.257293 21575 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:02.280488 21575 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:02.284123 21575 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:02.284392 21575 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:02.292953 21575 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40339
I20260811 22:14:02.293017 21658 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40339 every 8 connection(s)
I20260811 22:14:02.294092 21575 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:02.297062 21659 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:02.302429 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 21575
I20260811 22:14:02.302560 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:02.302883 28862 external_mini_cluster.cc:1428] Setting key dd88ddf3b776076f13922dc42603c4a3
I20260811 22:14:02.303145 21659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7: Bootstrap starting.
I20260811 22:14:02.305887 21659 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:02.306763 21659 log.cc:824] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:02.309017 21659 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7: No bootstrap required, opened a new log
Aug 11 22:14:02 dist-test-slave-vxj2 krb5kdc[21560](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486442, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:14:02.313591 21659 raft_consensus.cc:348] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "070f22a756ea42f6a69d9243eca694a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40339 } }
I20260811 22:14:02.313849 21659 raft_consensus.cc:374] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:02.313942 21659 raft_consensus.cc:729] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 070f22a756ea42f6a69d9243eca694a7, State: Initialized, Role: FOLLOWER
I20260811 22:14:02.314553 21659 consensus_queue.cc:260] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "070f22a756ea42f6a69d9243eca694a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40339 } }
I20260811 22:14:02.314805 21659 raft_consensus.cc:388] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:02.314925 21659 raft_consensus.cc:482] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:02.315038 21659 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:02.316449 21659 raft_consensus.cc:504] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "070f22a756ea42f6a69d9243eca694a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40339 } }
I20260811 22:14:02.316902 21659 leader_election.cc:302] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 070f22a756ea42f6a69d9243eca694a7; no voters:
I20260811 22:14:02.317428 21659 leader_election.cc:288] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:02.317682 21664 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:02.317976 21664 raft_consensus.cc:686] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [term 1 LEADER]: Becoming Leader. State: Replica: 070f22a756ea42f6a69d9243eca694a7, State: Running, Role: LEADER
I20260811 22:14:02.318367 21664 consensus_queue.cc:237] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "070f22a756ea42f6a69d9243eca694a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40339 } }
I20260811 22:14:02.319144 21659 sys_catalog.cc:564] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:02.320214 21666 sys_catalog.cc:455] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 070f22a756ea42f6a69d9243eca694a7. Latest consensus state: current_term: 1 leader_uuid: "070f22a756ea42f6a69d9243eca694a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "070f22a756ea42f6a69d9243eca694a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40339 } } }
I20260811 22:14:02.320289 21662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:02.304563 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:34247 (local address 127.0.0.1:40339)
0811 22:14:02.305047 (+ 484us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:02.305081 (+ 34us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:02.305122 (+ 41us) server_negotiation.cc:407] Connection header received
0811 22:14:02.305937 (+ 815us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:02.305957 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:02.306409 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:02.306735 (+ 326us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:02.307700 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:02.308715 (+ 1015us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:02.309720 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:02.310214 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:02.310417 (+ 203us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:02.313461 (+ 3044us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:02.313466 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:02.313482 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:02.313513 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:02.315868 (+ 2355us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:02.316482 (+ 614us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:02.316489 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:02.316496 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:02.316660 (+ 164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:02.317080 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:02.317084 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:02.317086 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:02.317771 (+ 685us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:02.318676 (+ 905us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:02.318872 (+ 196us) server_negotiation.cc:299] Negotiation successful
0811 22:14:02.319133 (+ 261us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":168,"threads_started":1}
I20260811 22:14:02.320345 21666 sys_catalog.cc:458] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:02.320706 21665 sys_catalog.cc:455] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "070f22a756ea42f6a69d9243eca694a7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "070f22a756ea42f6a69d9243eca694a7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40339 } } }
I20260811 22:14:02.320806 21665 sys_catalog.cc:458] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:02.322110 21674 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:02.324164 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:02.326031 21674 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:02.332674 21674 catalog_manager.cc:1347] Generated new cluster ID: 93837a11621b4d62968c7e23c4cf7bbc
I20260811 22:14:02.332773 21674 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:02.342571 21662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:02.326702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:50389 (local address 127.0.0.1:40339)
0811 22:14:02.328108 (+ 1406us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:02.328112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:02.328126 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:14:02.328159 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:02.328162 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:02.328204 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:02.328317 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:02.329085 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:02.330195 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:02.331419 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:02.331701 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:02.331834 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:02.338520 (+ 6686us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:02.338525 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:02.338528 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:02.338567 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:02.340450 (+ 1883us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:02.341082 (+ 632us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:02.341085 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:02.341087 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:02.341210 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:02.341586 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:02.341589 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:02.341591 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:02.341832 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:02.341983 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:02.342284 (+ 301us) server_negotiation.cc:299] Negotiation successful
0811 22:14:02.342394 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1336}
I20260811 22:14:02.360522 21674 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:02.361769 21674 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:02.365867 21674 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 070f22a756ea42f6a69d9243eca694a7: Generated new TSK 0
I20260811 22:14:02.366648 21674 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:02.380328 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 21575
W20260811 22:14:02.390679 21558 connection.cc:535] client connection to 127.0.0.1:40339 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40339): Connection reset by peer (error 104)
2026-08-11T22:14:02Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1575 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:02 dist-test-slave-vxj2 krb5kdc[21693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:02 dist-test-slave-vxj2 krb5kdc[21693](info): set up 2 sockets
Aug 11 22:14:02 dist-test-slave-vxj2 krb5kdc[21693](info): commencing operation
krb5kdc: starting...
W20260811 22:14:03.547849 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:03 dist-test-slave-vxj2 krb5kdc[21693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486443, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:14:03.689406 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41469
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42625
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:41469
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:03.810982 21709 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:03.811369 21709 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:03.811462 21709 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:03.815846 21709 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:03.815968 21709 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:03.816006 21709 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:03.816047 21709 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:03.816077 21709 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:03.822041 21709 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42625
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:41469
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:41469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21709
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:03.823361 21709 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:03.824545 21709 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:03.834328 21715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:03.834328 21717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:03.834328 21714 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:03.834571 21709 server_base.cc:1034] running on GCE node
I20260811 22:14:03.835232 21709 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:03.836438 21709 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:03.837652 21709 hybrid_clock.cc:648] HybridClock initialized: now 1786486443837633 us; error 33 us; skew 500 ppm
Aug 11 22:14:03 dist-test-slave-vxj2 krb5kdc[21693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486443, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:03.840873 21709 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:14:03.841958 21709 webserver.cc:460] Webserver started at http://127.0.0.1:35527/ using document root <none> and password file <none>
I20260811 22:14:03.842536 21709 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:03.842599 21709 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:03.842777 21709 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:03.844951 21709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "14e835d70a994a15af0a34788bc1f814"
format_stamp: "Formatted at 2026-08-11 22:14:03 on dist-test-slave-vxj2"
server_key: "d5534946aa20596f381d8bfcf74c8d45"
server_key_iv: "da80912cc6f16edcbafffd502f4ceb9e"
server_key_version: "encryptionkey@0"
I20260811 22:14:03.845616 21709 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "14e835d70a994a15af0a34788bc1f814"
format_stamp: "Formatted at 2026-08-11 22:14:03 on dist-test-slave-vxj2"
server_key: "d5534946aa20596f381d8bfcf74c8d45"
server_key_iv: "da80912cc6f16edcbafffd502f4ceb9e"
server_key_version: "encryptionkey@0"
I20260811 22:14:03.849821 21709 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:14:03.852507 21725 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:03.854033 21709 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:03.854206 21709 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "14e835d70a994a15af0a34788bc1f814"
format_stamp: "Formatted at 2026-08-11 22:14:03 on dist-test-slave-vxj2"
server_key: "d5534946aa20596f381d8bfcf74c8d45"
server_key_iv: "da80912cc6f16edcbafffd502f4ceb9e"
server_key_version: "encryptionkey@0"
I20260811 22:14:03.854349 21709 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:03.872138 21709 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:03.875733 21709 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:03.875969 21709 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:03.884299 21709 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:41469
I20260811 22:14:03.884330 21784 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:41469 every 8 connection(s)
I20260811 22:14:03.885363 21709 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:03.887647 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 21709
I20260811 22:14:03.887773 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:03.888067 28862 external_mini_cluster.cc:1428] Setting key ff79636c800a73451237a1d6dd66a76f
I20260811 22:14:03.888671 21786 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:03.895164 21786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814: Bootstrap starting.
I20260811 22:14:03.897554 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:03.889634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:42967 (local address 127.0.0.1:41469)
0811 22:14:03.890141 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:03.890157 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:03.890197 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:14:03.890875 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:03.890891 (+ 16us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:03.891252 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:03.891616 (+ 364us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:03.892032 (+ 416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:03.893029 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:03.894002 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:03.894502 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:03.894682 (+ 180us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:03.895730 (+ 1048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:03.895735 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:03.895749 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:03.896175 (+ 426us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:03.896231 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:03.896254 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:03.896398 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:03.896599 (+ 201us) server_negotiation.cc:299] Negotiation successful
0811 22:14:03.896746 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":156,"threads_started":1}
I20260811 22:14:03.897737 21786 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:03.898663 21786 log.cc:824] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:03.901119 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:03.901345 21786 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814: No bootstrap required, opened a new log
I20260811 22:14:03.905915 21786 raft_consensus.cc:348] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e835d70a994a15af0a34788bc1f814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41469 } }
I20260811 22:14:03.906236 21786 raft_consensus.cc:374] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:03.906318 21786 raft_consensus.cc:729] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 14e835d70a994a15af0a34788bc1f814, State: Initialized, Role: FOLLOWER
I20260811 22:14:03.906889 21786 consensus_queue.cc:260] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e835d70a994a15af0a34788bc1f814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41469 } }
I20260811 22:14:03.907128 21786 raft_consensus.cc:388] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:03.907238 21786 raft_consensus.cc:482] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:03.907366 21786 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:03.908367 21789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:03.903281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:59995 (local address 127.0.0.1:41469)
0811 22:14:03.903431 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:03.903435 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:03.903682 (+ 247us) server_negotiation.cc:407] Connection header received
0811 22:14:03.903841 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:03.903843 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:03.903923 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:03.903993 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:03.904309 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:03.904803 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:03.905948 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:03.906179 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:03.906301 (+ 122us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:03.907397 (+ 1096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:03.907420 (+ 23us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:03.907425 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:03.907446 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:03.907484 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:03.907503 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:03.907603 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:03.907849 (+ 246us) server_negotiation.cc:299] Negotiation successful
0811 22:14:03.907930 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260811 22:14:03.908969 21786 raft_consensus.cc:504] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e835d70a994a15af0a34788bc1f814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41469 } }
I20260811 22:14:03.909413 21786 leader_election.cc:302] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 14e835d70a994a15af0a34788bc1f814; no voters:
I20260811 22:14:03.909891 21786 leader_election.cc:288] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:03.910070 21799 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:03.910352 21799 raft_consensus.cc:686] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [term 1 LEADER]: Becoming Leader. State: Replica: 14e835d70a994a15af0a34788bc1f814, State: Running, Role: LEADER
I20260811 22:14:03.910799 21799 consensus_queue.cc:237] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e835d70a994a15af0a34788bc1f814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41469 } }
I20260811 22:14:03.911729 21786 sys_catalog.cc:564] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:03.912600 21803 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 14e835d70a994a15af0a34788bc1f814. Latest consensus state: current_term: 1 leader_uuid: "14e835d70a994a15af0a34788bc1f814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e835d70a994a15af0a34788bc1f814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41469 } } }
I20260811 22:14:03.912645 21801 sys_catalog.cc:455] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "14e835d70a994a15af0a34788bc1f814" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "14e835d70a994a15af0a34788bc1f814" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 41469 } } }
I20260811 22:14:03.912744 21803 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:03.912784 21801 sys_catalog.cc:458] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:03.913278 21812 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:03.917232 21812 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:03.923763 21812 catalog_manager.cc:1347] Generated new cluster ID: b6b6bd50d5604f208a2203a5a8b4a718
I20260811 22:14:03.923866 21812 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:03.935922 21812 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:03.937211 21812 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:03.943233 21812 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 14e835d70a994a15af0a34788bc1f814: Generated new TSK 0
I20260811 22:14:03.944068 21812 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:03.950088 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 21709
W20260811 22:14:03.960994 21691 connection.cc:535] client connection to 127.0.0.1:41469 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:41469): Connection reset by peer (error 104)
2026-08-11T22:14:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1570 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:04 dist-test-slave-vxj2 krb5kdc[21824](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:04 dist-test-slave-vxj2 krb5kdc[21824](info): set up 2 sockets
Aug 11 22:14:04 dist-test-slave-vxj2 krb5kdc[21824](info): commencing operation
krb5kdc: starting...
W20260811 22:14:05.131114 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.155s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:05 dist-test-slave-vxj2 krb5kdc[21824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486445, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:14:05.272243 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46411
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42503
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46411
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:05.401945 21839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:05.402390 21839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:05.402486 21839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:05.409090 21839 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:05.409294 21839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:05.409349 21839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:05.409389 21839 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:05.409426 21839 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:05.415880 21839 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42503
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46411
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46411
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:05.417248 21839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:05.418352 21839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:05.427752 21847 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:05.427776 21849 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:05.427767 21845 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:05.428061 21839 server_base.cc:1034] running on GCE node
I20260811 22:14:05.428905 21839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:05.430336 21839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:05.431564 21839 hybrid_clock.cc:648] HybridClock initialized: now 1786486445431536 us; error 53 us; skew 500 ppm
Aug 11 22:14:05 dist-test-slave-vxj2 krb5kdc[21824](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486445, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:05.434788 21839 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:14:05.435802 21839 webserver.cc:460] Webserver started at http://127.0.0.1:44147/ using document root <none> and password file <none>
I20260811 22:14:05.436404 21839 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:05.436511 21839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:05.436758 21839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:05.439009 21839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "5ef07868d85048ab8b9d5229e0a272b5"
format_stamp: "Formatted at 2026-08-11 22:14:05 on dist-test-slave-vxj2"
server_key: "8fa4ab65d1eab1793d2f2a5aa381d222"
server_key_iv: "a6f8c0fb16b1f1a5c2aa37b4c363cdd2"
server_key_version: "encryptionkey@0"
I20260811 22:14:05.439622 21839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "5ef07868d85048ab8b9d5229e0a272b5"
format_stamp: "Formatted at 2026-08-11 22:14:05 on dist-test-slave-vxj2"
server_key: "8fa4ab65d1eab1793d2f2a5aa381d222"
server_key_iv: "a6f8c0fb16b1f1a5c2aa37b4c363cdd2"
server_key_version: "encryptionkey@0"
I20260811 22:14:05.443764 21839 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:14:05.446573 21856 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:05.448284 21839 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:14:05.448575 21839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "5ef07868d85048ab8b9d5229e0a272b5"
format_stamp: "Formatted at 2026-08-11 22:14:05 on dist-test-slave-vxj2"
server_key: "8fa4ab65d1eab1793d2f2a5aa381d222"
server_key_iv: "a6f8c0fb16b1f1a5c2aa37b4c363cdd2"
server_key_version: "encryptionkey@0"
I20260811 22:14:05.448774 21839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:05.455197 21839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:05.458961 21839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:05.459201 21839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:05.467209 21839 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46411
I20260811 22:14:05.467228 21914 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46411 every 8 connection(s)
I20260811 22:14:05.468330 21839 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:05.470799 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 21839
I20260811 22:14:05.470958 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:05.471272 28862 external_mini_cluster.cc:1428] Setting key a58e814ffbc09b531705007089abf808
I20260811 22:14:05.472100 21915 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:05.476962 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:05.472753 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:60801 (local address 127.0.0.1:46411)
0811 22:14:05.473251 (+ 498us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:05.473268 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:05.473313 (+ 45us) server_negotiation.cc:407] Connection header received
0811 22:14:05.473960 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:05.473977 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:05.474262 (+ 285us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:05.474621 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:05.475550 (+ 929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:05.475555 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:05.475568 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:05.475579 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:05.475638 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:05.475659 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:05.475780 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:05.475917 (+ 137us) server_negotiation.cc:299] Negotiation successful
0811 22:14:05.476114 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":305,"thread_start_us":152,"threads_started":1}
I20260811 22:14:05.479607 21915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5: Bootstrap starting.
I20260811 22:14:05.480499 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:05.482578 21915 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:05.483754 21915 log.cc:824] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:05.485049 21918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:05.483096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:54481 (local address 127.0.0.1:46411)
0811 22:14:05.483252 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:05.483257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:05.483429 (+ 172us) server_negotiation.cc:407] Connection header received
0811 22:14:05.483538 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:05.483541 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:05.483587 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:05.483689 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:05.484588 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:05.484591 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:05.484599 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:05.484610 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:05.484635 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:05.484645 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:05.484720 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:05.484853 (+ 133us) server_negotiation.cc:299] Negotiation successful
0811 22:14:05.484891 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:14:05.486438 21915 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5: No bootstrap required, opened a new log
I20260811 22:14:05.490530 21915 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ef07868d85048ab8b9d5229e0a272b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46411 } }
I20260811 22:14:05.490926 21915 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:05.490999 21915 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ef07868d85048ab8b9d5229e0a272b5, State: Initialized, Role: FOLLOWER
I20260811 22:14:05.491591 21915 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ef07868d85048ab8b9d5229e0a272b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46411 } }
I20260811 22:14:05.491748 21915 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:05.491797 21915 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:05.491873 21915 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:05.492908 21915 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ef07868d85048ab8b9d5229e0a272b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46411 } }
I20260811 22:14:05.493321 21915 leader_election.cc:302] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5ef07868d85048ab8b9d5229e0a272b5; no voters:
I20260811 22:14:05.493695 21915 leader_election.cc:288] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:05.493841 21926 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:05.494127 21926 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [term 1 LEADER]: Becoming Leader. State: Replica: 5ef07868d85048ab8b9d5229e0a272b5, State: Running, Role: LEADER
I20260811 22:14:05.494493 21926 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ef07868d85048ab8b9d5229e0a272b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46411 } }
I20260811 22:14:05.495287 21915 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:05.496603 21927 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5ef07868d85048ab8b9d5229e0a272b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ef07868d85048ab8b9d5229e0a272b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46411 } } }
I20260811 22:14:05.496649 21928 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5ef07868d85048ab8b9d5229e0a272b5. Latest consensus state: current_term: 1 leader_uuid: "5ef07868d85048ab8b9d5229e0a272b5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5ef07868d85048ab8b9d5229e0a272b5" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46411 } } }
I20260811 22:14:05.496758 21927 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:05.496762 21928 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:05.497208 21939 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:05.500777 21939 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:05.507488 21939 catalog_manager.cc:1347] Generated new cluster ID: 4f62bcd7f07f40bda69b507cb3af500f
I20260811 22:14:05.507603 21939 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:05.517740 21939 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:05.518160 21939 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:05.527374 21939 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5ef07868d85048ab8b9d5229e0a272b5: Generated new TSK 0
I20260811 22:14:05.528342 21939 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:05.535419 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 21839
2026-08-11T22:14:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1583 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:05 dist-test-slave-vxj2 krb5kdc[21955](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:05 dist-test-slave-vxj2 krb5kdc[21955](info): set up 2 sockets
Aug 11 22:14:05 dist-test-slave-vxj2 krb5kdc[21955](info): commencing operation
krb5kdc: starting...
W20260811 22:14:06.677699 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:06 dist-test-slave-vxj2 krb5kdc[21955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:14:06.813900 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44575
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:41387
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44575
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:06.932771 21970 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:06.933077 21970 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:06.933125 21970 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:06.937486 21970 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:06.937569 21970 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:06.937589 21970 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:06.937608 21970 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:06.937623 21970 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:06.943576 21970 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:41387
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44575
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44575
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21970
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:06.944846 21970 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:06.946084 21970 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:06.956079 21975 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:06.956084 21978 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:06.956210 21970 server_base.cc:1034] running on GCE node
W20260811 22:14:06.956084 21976 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:06.957100 21970 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:06.958408 21970 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:06.959628 21970 hybrid_clock.cc:648] HybridClock initialized: now 1786486446959588 us; error 57 us; skew 500 ppm
Aug 11 22:14:06 dist-test-slave-vxj2 krb5kdc[21955](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486446, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:06.969244 21970 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:14:06.970337 21970 webserver.cc:460] Webserver started at http://127.0.0.1:42253/ using document root <none> and password file <none>
I20260811 22:14:06.970976 21970 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:06.971081 21970 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:06.971341 21970 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:06.973737 21970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "19bb88b1e81f4792911ede910d1c01c7"
format_stamp: "Formatted at 2026-08-11 22:14:06 on dist-test-slave-vxj2"
server_key: "5a1e603a78da4e57adee343634b4068d"
server_key_iv: "d2470f7c9238a42f5dd43cc8d601c12e"
server_key_version: "encryptionkey@0"
I20260811 22:14:06.974371 21970 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "19bb88b1e81f4792911ede910d1c01c7"
format_stamp: "Formatted at 2026-08-11 22:14:06 on dist-test-slave-vxj2"
server_key: "5a1e603a78da4e57adee343634b4068d"
server_key_iv: "d2470f7c9238a42f5dd43cc8d601c12e"
server_key_version: "encryptionkey@0"
I20260811 22:14:06.978439 21970 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260811 22:14:06.981208 21987 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:06.982722 21970 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:14:06.982993 21970 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "19bb88b1e81f4792911ede910d1c01c7"
format_stamp: "Formatted at 2026-08-11 22:14:06 on dist-test-slave-vxj2"
server_key: "5a1e603a78da4e57adee343634b4068d"
server_key_iv: "d2470f7c9238a42f5dd43cc8d601c12e"
server_key_version: "encryptionkey@0"
I20260811 22:14:06.983132 21970 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:07.006245 21970 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:07.010010 21970 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:07.010243 21970 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:07.018309 21970 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44575
I20260811 22:14:07.018357 22049 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44575 every 8 connection(s)
I20260811 22:14:07.019412 21970 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:07.022382 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 21970
I20260811 22:14:07.022575 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:07.022639 22050 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:07.022964 28862 external_mini_cluster.cc:1428] Setting key 70344a1052f0647d87c41e1c1e9e2ca7
I20260811 22:14:07.029691 22050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7: Bootstrap starting.
I20260811 22:14:07.032428 22050 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:07.033183 22050 log.cc:824] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:07.035358 22050 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7: No bootstrap required, opened a new log
Aug 11 22:14:07 dist-test-slave-vxj2 krb5kdc[21955](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486446, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260811 22:14:07.040043 22050 raft_consensus.cc:348] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19bb88b1e81f4792911ede910d1c01c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } }
I20260811 22:14:07.040366 22050 raft_consensus.cc:374] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:07.040446 22050 raft_consensus.cc:729] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 19bb88b1e81f4792911ede910d1c01c7, State: Initialized, Role: FOLLOWER
I20260811 22:14:07.041052 22050 consensus_queue.cc:260] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19bb88b1e81f4792911ede910d1c01c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } }
I20260811 22:14:07.041262 22050 raft_consensus.cc:388] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:07.041348 22050 raft_consensus.cc:482] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:07.041461 22050 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:07.042512 22050 raft_consensus.cc:504] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19bb88b1e81f4792911ede910d1c01c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } }
I20260811 22:14:07.042966 22050 leader_election.cc:302] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 19bb88b1e81f4792911ede910d1c01c7; no voters:
I20260811 22:14:07.043385 22050 leader_election.cc:288] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:07.043481 22056 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:07.043677 22056 raft_consensus.cc:686] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [term 1 LEADER]: Becoming Leader. State: Replica: 19bb88b1e81f4792911ede910d1c01c7, State: Running, Role: LEADER
I20260811 22:14:07.044068 22056 consensus_queue.cc:237] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19bb88b1e81f4792911ede910d1c01c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } }
I20260811 22:14:07.044106 22053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.024561 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:38347 (local address 127.0.0.1:44575)
0811 22:14:07.025067 (+ 506us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:07.025082 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:07.025120 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:14:07.026173 (+ 1053us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:07.026195 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:07.026595 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:07.026984 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:07.027846 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.028862 (+ 1016us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.029910 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.030608 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.030796 (+ 188us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.037313 (+ 6517us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:07.037321 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:07.037341 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:07.037383 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:07.040165 (+ 2782us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.040932 (+ 767us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.040937 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.040943 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.041099 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.041635 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.041638 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.041639 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.042110 (+ 471us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:07.042384 (+ 274us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:07.042731 (+ 347us) server_negotiation.cc:299] Negotiation successful
0811 22:14:07.042985 (+ 254us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":351,"thread_start_us":171,"threads_started":1}
I20260811 22:14:07.044857 22050 sys_catalog.cc:564] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:07.046906 22057 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "19bb88b1e81f4792911ede910d1c01c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19bb88b1e81f4792911ede910d1c01c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } } }
I20260811 22:14:07.047075 22057 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:07.047475 22072 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:07.047787 22058 sys_catalog.cc:455] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 19bb88b1e81f4792911ede910d1c01c7. Latest consensus state: current_term: 1 leader_uuid: "19bb88b1e81f4792911ede910d1c01c7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "19bb88b1e81f4792911ede910d1c01c7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44575 } } }
I20260811 22:14:07.048015 22058 sys_catalog.cc:458] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:07.048231 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:07.051270 22072 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:07.058272 22072 catalog_manager.cc:1347] Generated new cluster ID: 4482aed39a0744e0b08451168e68bcb9
I20260811 22:14:07.058352 22072 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:07.064060 22053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:07.050370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:45743 (local address 127.0.0.1:44575)
0811 22:14:07.050492 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:07.050497 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:07.051694 (+ 1197us) server_negotiation.cc:407] Connection header received
0811 22:14:07.051841 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:07.051845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:07.051907 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:07.052005 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:07.052917 (+ 912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.053787 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.054901 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:07.055278 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:07.055391 (+ 113us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:07.057056 (+ 1665us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:07.057062 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:07.057070 (+ 8us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:07.057096 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:07.061977 (+ 4881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.062685 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.062688 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.062690 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.062782 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:07.063143 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:07.063146 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:07.063148 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:07.063356 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:07.063502 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:07.063787 (+ 285us) server_negotiation.cc:299] Negotiation successful
0811 22:14:07.063889 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:14:07.075647 22072 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:07.077136 22072 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:07.088382 22072 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 19bb88b1e81f4792911ede910d1c01c7: Generated new TSK 0
I20260811 22:14:07.089566 22072 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:07.104110 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 21970
W20260811 22:14:07.114318 21951 connection.cc:535] client connection to 127.0.0.1:44575 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44575): Connection reset by peer (error 104)
2026-08-11T22:14:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:07 dist-test-slave-vxj2 krb5kdc[22083](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:07 dist-test-slave-vxj2 krb5kdc[22083](info): set up 2 sockets
Aug 11 22:14:07 dist-test-slave-vxj2 krb5kdc[22083](info): commencing operation
krb5kdc: starting...
W20260811 22:14:08.270555 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:08 dist-test-slave-vxj2 krb5kdc[22083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486448, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:14:08.412729 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43465
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39295
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43465
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:08.532658 22098 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:08.533028 22098 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:08.533123 22098 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:08.537365 22098 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:08.537477 22098 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:08.537518 22098 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:08.537554 22098 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:08.537596 22098 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:08.543311 22098 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39295
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43465
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22098
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:08.544552 22098 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:08.545733 22098 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:08.554924 22109 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:08.555074 22098 server_base.cc:1034] running on GCE node
W20260811 22:14:08.554929 22106 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:08.554929 22104 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:08.555912 22098 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:08.557106 22098 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:08.558346 22098 hybrid_clock.cc:648] HybridClock initialized: now 1786486448558323 us; error 49 us; skew 500 ppm
Aug 11 22:14:08 dist-test-slave-vxj2 krb5kdc[22083](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486448, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:08.561463 22098 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:14:08.562475 22098 webserver.cc:460] Webserver started at http://127.0.0.1:37025/ using document root <none> and password file <none>
I20260811 22:14:08.563073 22098 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:08.563169 22098 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:08.563412 22098 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:08.565621 22098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "900cca2a0058440a854196be542981b0"
format_stamp: "Formatted at 2026-08-11 22:14:08 on dist-test-slave-vxj2"
server_key: "f8ef777edd5928ad46eac6ae3e642e95"
server_key_iv: "35adc5e0511ccff39244a7a168c629cd"
server_key_version: "encryptionkey@0"
I20260811 22:14:08.566251 22098 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "900cca2a0058440a854196be542981b0"
format_stamp: "Formatted at 2026-08-11 22:14:08 on dist-test-slave-vxj2"
server_key: "f8ef777edd5928ad46eac6ae3e642e95"
server_key_iv: "35adc5e0511ccff39244a7a168c629cd"
server_key_version: "encryptionkey@0"
I20260811 22:14:08.570271 22098 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:14:08.572942 22118 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:08.574486 22098 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:14:08.574677 22098 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "900cca2a0058440a854196be542981b0"
format_stamp: "Formatted at 2026-08-11 22:14:08 on dist-test-slave-vxj2"
server_key: "f8ef777edd5928ad46eac6ae3e642e95"
server_key_iv: "35adc5e0511ccff39244a7a168c629cd"
server_key_version: "encryptionkey@0"
I20260811 22:14:08.574815 22098 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:08.593758 22098 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:08.597466 22098 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:08.597715 22098 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:08.606089 22098 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43465
I20260811 22:14:08.606139 22179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43465 every 8 connection(s)
I20260811 22:14:08.607241 22098 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:08.610689 22181 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:08.610824 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 22098
I20260811 22:14:08.610960 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:08.611312 28862 external_mini_cluster.cc:1428] Setting key d2c55d54f77302876cc0ec84144e04bf
I20260811 22:14:08.619037 22181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0: Bootstrap starting.
I20260811 22:14:08.622186 22181 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:08.622933 22185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:08.612904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:44097 (local address 127.0.0.1:43465)
0811 22:14:08.613407 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:08.613431 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:08.613477 (+ 46us) server_negotiation.cc:407] Connection header received
0811 22:14:08.614690 (+ 1213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:08.614720 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:08.615214 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:08.615631 (+ 417us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:08.616084 (+ 453us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:08.617671 (+ 1587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:08.618771 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:08.619481 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:08.619659 (+ 178us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:08.620706 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:08.620713 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:08.620730 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:08.621304 (+ 574us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:08.621378 (+ 74us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:08.621413 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:08.621553 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:08.621756 (+ 203us) server_negotiation.cc:299] Negotiation successful
0811 22:14:08.621908 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":188,"threads_started":1}
I20260811 22:14:08.623142 22181 log.cc:824] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:08.625696 22181 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0: No bootstrap required, opened a new log
I20260811 22:14:08.626547 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:08.630887 22181 raft_consensus.cc:348] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900cca2a0058440a854196be542981b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43465 } }
I20260811 22:14:08.631179 22181 raft_consensus.cc:374] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:08.631227 22181 raft_consensus.cc:729] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 900cca2a0058440a854196be542981b0, State: Initialized, Role: FOLLOWER
I20260811 22:14:08.631793 22181 consensus_queue.cc:260] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900cca2a0058440a854196be542981b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43465 } }
I20260811 22:14:08.631978 22181 raft_consensus.cc:388] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:08.632057 22181 raft_consensus.cc:482] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:08.632189 22181 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:08.633701 22185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:08.628987 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:57349 (local address 127.0.0.1:43465)
0811 22:14:08.629138 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:08.629145 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:08.629316 (+ 171us) server_negotiation.cc:407] Connection header received
0811 22:14:08.629417 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:08.629423 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:08.629480 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:08.629576 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:08.629958 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:08.630736 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:08.631694 (+ 958us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:08.632051 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:08.632170 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:08.633136 (+ 966us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:08.633140 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:08.633142 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:08.633190 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:08.633226 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:08.633245 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:08.633349 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:08.633479 (+ 130us) server_negotiation.cc:299] Negotiation successful
0811 22:14:08.633527 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260811 22:14:08.633670 22181 raft_consensus.cc:504] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900cca2a0058440a854196be542981b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43465 } }
I20260811 22:14:08.634069 22181 leader_election.cc:302] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 900cca2a0058440a854196be542981b0; no voters:
I20260811 22:14:08.634462 22181 leader_election.cc:288] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:08.634639 22193 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:08.635015 22193 raft_consensus.cc:686] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [term 1 LEADER]: Becoming Leader. State: Replica: 900cca2a0058440a854196be542981b0, State: Running, Role: LEADER
I20260811 22:14:08.635380 22193 consensus_queue.cc:237] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900cca2a0058440a854196be542981b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43465 } }
I20260811 22:14:08.635865 22181 sys_catalog.cc:564] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:08.637251 22195 sys_catalog.cc:455] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "900cca2a0058440a854196be542981b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900cca2a0058440a854196be542981b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43465 } } }
I20260811 22:14:08.637408 22195 sys_catalog.cc:458] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:08.637267 22197 sys_catalog.cc:455] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 900cca2a0058440a854196be542981b0. Latest consensus state: current_term: 1 leader_uuid: "900cca2a0058440a854196be542981b0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "900cca2a0058440a854196be542981b0" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43465 } } }
I20260811 22:14:08.637738 22197 sys_catalog.cc:458] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0 [sys.catalog]: This master's current role is: LEADER
W20260811 22:14:08.641292 22211 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:14:08.641373 22211 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:14:08.641470 22206 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:08.642230 22206 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:08.648263 22206 catalog_manager.cc:1347] Generated new cluster ID: 306438f57e1048239ce899038d9e59fa
I20260811 22:14:08.648356 22206 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:08.669292 22206 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:08.670315 22206 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:08.677563 22206 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 900cca2a0058440a854196be542981b0: Generated new TSK 0
I20260811 22:14:08.678434 22206 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:08.689754 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 22098
W20260811 22:14:08.700906 22081 connection.cc:535] client connection to 127.0.0.1:43465 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43465): Connection reset by peer (error 104)
2026-08-11T22:14:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1586 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:08 dist-test-slave-vxj2 krb5kdc[22218](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:08 dist-test-slave-vxj2 krb5kdc[22218](info): set up 2 sockets
Aug 11 22:14:08 dist-test-slave-vxj2 krb5kdc[22218](info): commencing operation
krb5kdc: starting...
W20260811 22:14:09.840910 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:09 dist-test-slave-vxj2 krb5kdc[22218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486449, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260811 22:14:09.978499 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40025
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35747
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40025
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:10.099489 22232 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:10.099817 22232 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:10.100085 22232 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:10.104473 22232 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:10.104568 22232 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:10.104589 22232 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:10.104607 22232 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:10.104624 22232 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:10.110689 22232 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35747
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40025
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40025
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22232
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:10.111943 22232 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:10.113106 22232 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:10.122180 22243 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:10.122180 22240 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:10.122197 22239 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:10.122540 22232 server_base.cc:1034] running on GCE node
I20260811 22:14:10.123319 22232 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:10.124567 22232 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:10.125725 22232 hybrid_clock.cc:648] HybridClock initialized: now 1786486450125711 us; error 29 us; skew 500 ppm
Aug 11 22:14:10 dist-test-slave-vxj2 krb5kdc[22218](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486450, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:10.129339 22232 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260811 22:14:10.130481 22232 webserver.cc:460] Webserver started at http://127.0.0.1:37797/ using document root <none> and password file <none>
I20260811 22:14:10.131167 22232 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:10.131244 22232 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:10.131497 22232 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:10.133839 22232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "8f026ac26eae4bf4a9f82d991ff8960a"
format_stamp: "Formatted at 2026-08-11 22:14:10 on dist-test-slave-vxj2"
server_key: "8b54a659b38bc17f38aba8c3a77dd5d2"
server_key_iv: "052cc21dcf441ec62514182cc5959ca1"
server_key_version: "encryptionkey@0"
I20260811 22:14:10.134473 22232 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "8f026ac26eae4bf4a9f82d991ff8960a"
format_stamp: "Formatted at 2026-08-11 22:14:10 on dist-test-slave-vxj2"
server_key: "8b54a659b38bc17f38aba8c3a77dd5d2"
server_key_iv: "052cc21dcf441ec62514182cc5959ca1"
server_key_version: "encryptionkey@0"
I20260811 22:14:10.138655 22232 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:14:10.141592 22250 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:10.143102 22232 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260811 22:14:10.143301 22232 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "8f026ac26eae4bf4a9f82d991ff8960a"
format_stamp: "Formatted at 2026-08-11 22:14:10 on dist-test-slave-vxj2"
server_key: "8b54a659b38bc17f38aba8c3a77dd5d2"
server_key_iv: "052cc21dcf441ec62514182cc5959ca1"
server_key_version: "encryptionkey@0"
I20260811 22:14:10.143441 22232 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:10.148943 22232 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:10.153026 22232 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:10.153422 22232 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:10.161484 22232 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40025
I20260811 22:14:10.161497 22314 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40025 every 8 connection(s)
I20260811 22:14:10.162598 22232 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:10.165937 22315 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:10.166520 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 22232
I20260811 22:14:10.166678 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:10.166975 28862 external_mini_cluster.cc:1428] Setting key a17e8c7399a1eb55128182e98d57fff8
I20260811 22:14:10.172708 22318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:10.168611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:39163 (local address 127.0.0.1:40025)
0811 22:14:10.169138 (+ 527us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:10.169209 (+ 71us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:10.169249 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:14:10.169951 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:10.169970 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:10.170267 (+ 297us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:10.170618 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:10.171478 (+ 860us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:10.171483 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:10.171495 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:10.171506 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:10.171564 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:10.171585 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:10.171742 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:10.171814 (+ 72us) server_negotiation.cc:299] Negotiation successful
0811 22:14:10.171994 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":161,"threads_started":1}
I20260811 22:14:10.172890 22315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a: Bootstrap starting.
I20260811 22:14:10.175525 22315 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:10.175663 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:10.176357 22315 log.cc:824] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:10.178820 22315 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a: No bootstrap required, opened a new log
I20260811 22:14:10.179625 22318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:10.177868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.142:60633 (local address 127.0.0.1:40025)
0811 22:14:10.178003 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:10.178008 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:10.178184 (+ 176us) server_negotiation.cc:407] Connection header received
0811 22:14:10.178326 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:10.178329 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:10.178361 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:10.178427 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:10.179223 (+ 796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:10.179226 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:10.179228 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:10.179239 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:10.179264 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:10.179274 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:10.179348 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:10.179454 (+ 106us) server_negotiation.cc:299] Negotiation successful
0811 22:14:10.179489 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:14:10.183038 22315 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f026ac26eae4bf4a9f82d991ff8960a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40025 } }
I20260811 22:14:10.183292 22315 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:10.183355 22315 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8f026ac26eae4bf4a9f82d991ff8960a, State: Initialized, Role: FOLLOWER
I20260811 22:14:10.183796 22315 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f026ac26eae4bf4a9f82d991ff8960a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40025 } }
I20260811 22:14:10.183943 22315 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:10.184032 22315 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:10.184152 22315 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:10.185199 22315 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f026ac26eae4bf4a9f82d991ff8960a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40025 } }
I20260811 22:14:10.185576 22315 leader_election.cc:302] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8f026ac26eae4bf4a9f82d991ff8960a; no voters:
I20260811 22:14:10.185916 22315 leader_election.cc:288] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:10.186084 22327 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:10.186338 22327 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [term 1 LEADER]: Becoming Leader. State: Replica: 8f026ac26eae4bf4a9f82d991ff8960a, State: Running, Role: LEADER
I20260811 22:14:10.186728 22327 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f026ac26eae4bf4a9f82d991ff8960a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40025 } }
I20260811 22:14:10.187274 22315 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:10.188694 22328 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8f026ac26eae4bf4a9f82d991ff8960a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f026ac26eae4bf4a9f82d991ff8960a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40025 } } }
I20260811 22:14:10.188694 22329 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8f026ac26eae4bf4a9f82d991ff8960a. Latest consensus state: current_term: 1 leader_uuid: "8f026ac26eae4bf4a9f82d991ff8960a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8f026ac26eae4bf4a9f82d991ff8960a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40025 } } }
I20260811 22:14:10.188853 22329 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:10.188853 22328 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:10.189338 22336 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:10.192828 22336 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:10.198841 22336 catalog_manager.cc:1347] Generated new cluster ID: 196f4726dc7d4a4bbf50e4c36e14a418
I20260811 22:14:10.198935 22336 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:10.218691 22336 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:10.219142 22336 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:10.224524 22336 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8f026ac26eae4bf4a9f82d991ff8960a: Generated new TSK 0
I20260811 22:14:10.225543 22336 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:10.239588 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 22232
2026-08-11T22:14:10Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:10 dist-test-slave-vxj2 krb5kdc[22352](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:10 dist-test-slave-vxj2 krb5kdc[22352](info): set up 2 sockets
Aug 11 22:14:10 dist-test-slave-vxj2 krb5kdc[22352](info): commencing operation
krb5kdc: starting...
W20260811 22:14:11.391295 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:11 dist-test-slave-vxj2 krb5kdc[22352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:11Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:11Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:14:11.528103 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:43083
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:37329
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:43083
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:11.649818 22365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:11.650179 22365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:11.650261 22365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:11.654525 22365 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:11.654641 22365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:11.654677 22365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:11.654716 22365 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:11.654757 22365 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:11.660485 22365 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:37329
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:43083
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:43083
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:11.661846 22365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:11.663062 22365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:11.672902 22370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:11.672818 22373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:11.672808 22371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:11.673466 22365 server_base.cc:1034] running on GCE node
I20260811 22:14:11.674261 22365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:11.675653 22365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:11.676892 22365 hybrid_clock.cc:648] HybridClock initialized: now 1786486451676856 us; error 52 us; skew 500 ppm
Aug 11 22:14:11 dist-test-slave-vxj2 krb5kdc[22352](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486451, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:11.680127 22365 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:14:11.681274 22365 webserver.cc:460] Webserver started at http://127.28.47.190:37339/ using document root <none> and password file <none>
I20260811 22:14:11.681916 22365 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:11.681989 22365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:11.682237 22365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:11.684508 22365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "da3fe5c07ebc4cc895579eb8b7dec36c"
format_stamp: "Formatted at 2026-08-11 22:14:11 on dist-test-slave-vxj2"
server_key: "76f9c955fb9a4dcbe418f97a3404afb0"
server_key_iv: "54cdc361c5ef08b18f8d12d4ea334ac8"
server_key_version: "encryptionkey@0"
I20260811 22:14:11.685109 22365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "da3fe5c07ebc4cc895579eb8b7dec36c"
format_stamp: "Formatted at 2026-08-11 22:14:11 on dist-test-slave-vxj2"
server_key: "76f9c955fb9a4dcbe418f97a3404afb0"
server_key_iv: "54cdc361c5ef08b18f8d12d4ea334ac8"
server_key_version: "encryptionkey@0"
I20260811 22:14:11.689406 22365 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:14:11.691974 22380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:11.693560 22365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:11.693711 22365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "da3fe5c07ebc4cc895579eb8b7dec36c"
format_stamp: "Formatted at 2026-08-11 22:14:11 on dist-test-slave-vxj2"
server_key: "76f9c955fb9a4dcbe418f97a3404afb0"
server_key_iv: "54cdc361c5ef08b18f8d12d4ea334ac8"
server_key_version: "encryptionkey@0"
I20260811 22:14:11.693835 22365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:11.712815 22365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:11.716153 22365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:11.716398 22365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:11.724298 22365 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:43083
I20260811 22:14:11.724342 22438 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:43083 every 8 connection(s)
I20260811 22:14:11.725409 22365 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:11.726512 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 22365
I20260811 22:14:11.726678 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:11.727029 28862 external_mini_cluster.cc:1428] Setting key 5cd3e37fd1b067e1ce32d3501e2e859a
I20260811 22:14:11.729292 22440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:11.735759 22440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c: Bootstrap starting.
Aug 11 22:14:11 dist-test-slave-vxj2 krb5kdc[22352](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486451, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:14:11.738744 22440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:11.739605 22440 log.cc:824] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:11.742062 22440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c: No bootstrap required, opened a new log
I20260811 22:14:11.742758 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:11.728606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41049 (local address 127.28.47.190:43083)
0811 22:14:11.729145 (+ 539us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:11.729221 (+ 76us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:11.729271 (+ 50us) server_negotiation.cc:407] Connection header received
0811 22:14:11.730044 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:11.730065 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:11.730432 (+ 367us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:11.730817 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:11.731732 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:11.732782 (+ 1050us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:11.733735 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:11.734238 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:11.734417 (+ 179us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:11.737553 (+ 3136us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:11.737558 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:11.737573 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:11.737607 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:11.739511 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:11.740119 (+ 608us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:11.740125 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:11.740130 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:11.740257 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:11.740608 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:11.740615 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:11.740617 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:11.741065 (+ 448us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:11.741350 (+ 285us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:11.741682 (+ 332us) server_negotiation.cc:299] Negotiation successful
0811 22:14:11.741928 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":387,"thread_start_us":187,"threads_started":1}
I20260811 22:14:11.746093 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:11.747001 22440 raft_consensus.cc:348] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da3fe5c07ebc4cc895579eb8b7dec36c" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43083 } }
I20260811 22:14:11.747323 22440 raft_consensus.cc:374] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:11.747403 22440 raft_consensus.cc:729] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: da3fe5c07ebc4cc895579eb8b7dec36c, State: Initialized, Role: FOLLOWER
I20260811 22:14:11.747934 22440 consensus_queue.cc:260] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da3fe5c07ebc4cc895579eb8b7dec36c" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43083 } }
I20260811 22:14:11.748111 22440 raft_consensus.cc:388] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:11.748214 22440 raft_consensus.cc:482] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:11.748358 22440 raft_consensus.cc:3037] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:11.749570 22440 raft_consensus.cc:504] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da3fe5c07ebc4cc895579eb8b7dec36c" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43083 } }
I20260811 22:14:11.749987 22440 leader_election.cc:302] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: da3fe5c07ebc4cc895579eb8b7dec36c; no voters:
I20260811 22:14:11.750401 22440 leader_election.cc:288] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:11.750484 22451 raft_consensus.cc:2781] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:11.750720 22451 raft_consensus.cc:686] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [term 1 LEADER]: Becoming Leader. State: Replica: da3fe5c07ebc4cc895579eb8b7dec36c, State: Running, Role: LEADER
I20260811 22:14:11.751174 22451 consensus_queue.cc:237] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da3fe5c07ebc4cc895579eb8b7dec36c" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43083 } }
I20260811 22:14:11.752054 22440 sys_catalog.cc:564] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:11.753141 22453 sys_catalog.cc:455] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "da3fe5c07ebc4cc895579eb8b7dec36c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da3fe5c07ebc4cc895579eb8b7dec36c" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43083 } } }
I20260811 22:14:11.753623 22453 sys_catalog.cc:458] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:11.753491 22452 sys_catalog.cc:455] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [sys.catalog]: SysCatalogTable state changed. Reason: New leader da3fe5c07ebc4cc895579eb8b7dec36c. Latest consensus state: current_term: 1 leader_uuid: "da3fe5c07ebc4cc895579eb8b7dec36c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "da3fe5c07ebc4cc895579eb8b7dec36c" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 43083 } } }
I20260811 22:14:11.753986 22452 sys_catalog.cc:458] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:11.754315 22462 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:11.758064 22462 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:11.758303 22443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:11.748209 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40833 (local address 127.28.47.190:43083)
0811 22:14:11.748350 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:11.748354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:11.748601 (+ 247us) server_negotiation.cc:407] Connection header received
0811 22:14:11.748769 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:11.748772 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:11.748824 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:11.748906 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:11.749690 (+ 784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:11.750344 (+ 654us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:11.751359 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:11.751675 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:11.751811 (+ 136us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:11.754283 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:11.754291 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:11.754297 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:11.754338 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:11.756117 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:11.756705 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:11.756709 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:11.756716 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:11.756862 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:11.757276 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:11.757283 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:11.757286 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:11.757497 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:11.757677 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:11.758003 (+ 326us) server_negotiation.cc:299] Negotiation successful
0811 22:14:11.758118 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260811 22:14:11.766074 22462 catalog_manager.cc:1347] Generated new cluster ID: 067e74a11bb647948d34ad148f105c72
I20260811 22:14:11.766180 22462 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:11.776872 22462 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:11.778000 22462 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:11.786621 22462 catalog_manager.cc:5868] T 00000000000000000000000000000000 P da3fe5c07ebc4cc895579eb8b7dec36c: Generated new TSK 0
I20260811 22:14:11.787454 22462 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:11.793671 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 22365
W20260811 22:14:11.802910 22350 connection.cc:535] client connection to 127.28.47.190:43083 recv error: Network error: failed to read from TLS socket (remote: 127.28.47.190:43083): Connection reset by peer (error 104)
2026-08-11T22:14:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1554 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:11 dist-test-slave-vxj2 krb5kdc[22478](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:11 dist-test-slave-vxj2 krb5kdc[22478](info): set up 2 sockets
Aug 11 22:14:11 dist-test-slave-vxj2 krb5kdc[22478](info): commencing operation
krb5kdc: starting...
W20260811 22:14:12.942597 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.007s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:13 dist-test-slave-vxj2 krb5kdc[22478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486453, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:14:13.081234 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:36735
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:45001
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:36735
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:13.207388 22493 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:13.207764 22493 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:13.207865 22493 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:13.212097 22493 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:13.212217 22493 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:13.212261 22493 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:13.212324 22493 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:13.212368 22493 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:13.218050 22493 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:45001
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:36735
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:36735
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22493
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:13.219379 22493 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:13.220561 22493 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:13.230362 22500 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:13.230404 22502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:13.230367 22499 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:13.230600 22493 server_base.cc:1034] running on GCE node
I20260811 22:14:13.231418 22493 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:13.232635 22493 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:13.233810 22493 hybrid_clock.cc:648] HybridClock initialized: now 1786486453233779 us; error 46 us; skew 500 ppm
Aug 11 22:14:13 dist-test-slave-vxj2 krb5kdc[22478](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486453, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:13.241293 22493 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:14:13.242265 22493 webserver.cc:460] Webserver started at http://127.28.47.190:37869/ using document root <none> and password file <none>
I20260811 22:14:13.242835 22493 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:13.242897 22493 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:13.243073 22493 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:13.245210 22493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "e5e7f2cfb9b74baab54536dcba6a70fb"
format_stamp: "Formatted at 2026-08-11 22:14:13 on dist-test-slave-vxj2"
server_key: "7b084345184c1321f896806741c0536f"
server_key_iv: "fcf2c384071477cc41d2240e6d6c30b4"
server_key_version: "encryptionkey@0"
I20260811 22:14:13.245770 22493 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "e5e7f2cfb9b74baab54536dcba6a70fb"
format_stamp: "Formatted at 2026-08-11 22:14:13 on dist-test-slave-vxj2"
server_key: "7b084345184c1321f896806741c0536f"
server_key_iv: "fcf2c384071477cc41d2240e6d6c30b4"
server_key_version: "encryptionkey@0"
I20260811 22:14:13.250108 22493 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:14:13.253830 22511 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:13.255764 22493 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260811 22:14:13.255955 22493 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "e5e7f2cfb9b74baab54536dcba6a70fb"
format_stamp: "Formatted at 2026-08-11 22:14:13 on dist-test-slave-vxj2"
server_key: "7b084345184c1321f896806741c0536f"
server_key_iv: "fcf2c384071477cc41d2240e6d6c30b4"
server_key_version: "encryptionkey@0"
I20260811 22:14:13.256108 22493 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:13.275493 22493 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:13.280263 22493 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:13.280527 22493 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:13.291610 22573 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:36735 every 8 connection(s)
I20260811 22:14:13.291632 22493 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:36735
I20260811 22:14:13.293311 22493 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:13.293759 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 22493
I20260811 22:14:13.293915 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:13.294256 28862 external_mini_cluster.cc:1428] Setting key 5122696f3266390bd2bcaa4d6bea7945
I20260811 22:14:13.297000 22574 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:13.303539 22574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb: Bootstrap starting.
I20260811 22:14:13.306847 22574 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:13.307086 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:13.295821 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45697 (local address 127.28.47.190:36735)
0811 22:14:13.296384 (+ 563us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:13.296409 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:13.296462 (+ 53us) server_negotiation.cc:407] Connection header received
0811 22:14:13.297731 (+ 1269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:13.297763 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:13.298380 (+ 617us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:13.298835 (+ 455us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:13.299340 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:13.301214 (+ 1874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:13.302552 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:13.303427 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:13.303638 (+ 211us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:13.304685 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:13.304692 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:13.304735 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:13.305349 (+ 614us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:13.305428 (+ 79us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:13.305463 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:13.305606 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:13.305872 (+ 266us) server_negotiation.cc:299] Negotiation successful
0811 22:14:13.306026 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":158,"threads_started":1}
I20260811 22:14:13.307834 22574 log.cc:824] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:13.310398 22574 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb: No bootstrap required, opened a new log
I20260811 22:14:13.310845 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:13.315009 22574 raft_consensus.cc:348] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e7f2cfb9b74baab54536dcba6a70fb" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36735 } }
I20260811 22:14:13.315265 22574 raft_consensus.cc:374] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:13.315301 22574 raft_consensus.cc:729] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e5e7f2cfb9b74baab54536dcba6a70fb, State: Initialized, Role: FOLLOWER
I20260811 22:14:13.315864 22574 consensus_queue.cc:260] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e7f2cfb9b74baab54536dcba6a70fb" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36735 } }
I20260811 22:14:13.316133 22574 raft_consensus.cc:388] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:13.316231 22574 raft_consensus.cc:482] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:13.316417 22574 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:13.317626 22574 raft_consensus.cc:504] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e7f2cfb9b74baab54536dcba6a70fb" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36735 } }
I20260811 22:14:13.318045 22574 leader_election.cc:302] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e5e7f2cfb9b74baab54536dcba6a70fb; no voters:
I20260811 22:14:13.318418 22574 leader_election.cc:288] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:13.318553 22586 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:13.318894 22586 raft_consensus.cc:686] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [term 1 LEADER]: Becoming Leader. State: Replica: e5e7f2cfb9b74baab54536dcba6a70fb, State: Running, Role: LEADER
I20260811 22:14:13.319348 22586 consensus_queue.cc:237] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e7f2cfb9b74baab54536dcba6a70fb" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36735 } }
I20260811 22:14:13.320254 22574 sys_catalog.cc:564] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:13.320779 22578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:13.313272 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46929 (local address 127.28.47.190:36735)
0811 22:14:13.313467 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:13.313473 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:13.315674 (+ 2201us) server_negotiation.cc:407] Connection header received
0811 22:14:13.315829 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:13.315836 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:13.315901 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:13.315995 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:13.316325 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:13.317346 (+ 1021us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:13.318576 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:13.318909 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:13.318996 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:13.320168 (+ 1172us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:13.320171 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:13.320174 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:13.320188 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:13.320216 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:13.320229 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:13.320298 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:13.320580 (+ 282us) server_negotiation.cc:299] Negotiation successful
0811 22:14:13.320625 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260811 22:14:13.320982 22586 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e5e7f2cfb9b74baab54536dcba6a70fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e7f2cfb9b74baab54536dcba6a70fb" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36735 } } }
I20260811 22:14:13.321233 22586 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:13.321523 22586 sys_catalog.cc:455] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [sys.catalog]: SysCatalogTable state changed. Reason: New leader e5e7f2cfb9b74baab54536dcba6a70fb. Latest consensus state: current_term: 1 leader_uuid: "e5e7f2cfb9b74baab54536dcba6a70fb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e5e7f2cfb9b74baab54536dcba6a70fb" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36735 } } }
I20260811 22:14:13.321646 22586 sys_catalog.cc:458] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:13.321703 22594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:13.325259 22594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:13.332145 22594 catalog_manager.cc:1347] Generated new cluster ID: a37a15d2761b4dce8535910515e0c8e6
I20260811 22:14:13.332324 22594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:13.361022 22594 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:13.362735 22594 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:13.380152 22594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e5e7f2cfb9b74baab54536dcba6a70fb: Generated new TSK 0
I20260811 22:14:13.381366 22594 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:13.389539 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 22493
W20260811 22:14:13.400246 22476 connection.cc:535] client connection to 127.28.47.190:36735 recv error: Network error: failed to read from TLS socket (remote: 127.28.47.190:36735): Connection reset by peer (error 104)
2026-08-11T22:14:13Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1597 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:13 dist-test-slave-vxj2 krb5kdc[22608](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:13 dist-test-slave-vxj2 krb5kdc[22608](info): set up 2 sockets
Aug 11 22:14:13 dist-test-slave-vxj2 krb5kdc[22608](info): commencing operation
krb5kdc: starting...
W20260811 22:14:14.549611 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.006s sys 0.001s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:14 dist-test-slave-vxj2 krb5kdc[22608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486454, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:14Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:14:14.684770 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:34055
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:36317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:34055
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:14.802452 22624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:14.802807 22624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:14.802907 22624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:14.807169 22624 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:14.807278 22624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:14.807313 22624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:14.807353 22624 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:14.807391 22624 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:14.813905 22624 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:36317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:34055
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:34055
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:14.815186 22624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:14.816318 22624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:14.825495 22633 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:14.825994 22624 server_base.cc:1034] running on GCE node
W20260811 22:14:14.825495 22636 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:14.825578 22632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:14.826788 22624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:14.828061 22624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:14.829231 22624 hybrid_clock.cc:648] HybridClock initialized: now 1786486454829208 us; error 34 us; skew 500 ppm
Aug 11 22:14:14 dist-test-slave-vxj2 krb5kdc[22608](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486454, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:14.832291 22624 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:14:14.833316 22624 webserver.cc:460] Webserver started at http://127.28.47.190:35577/ using document root <none> and password file <none>
I20260811 22:14:14.833869 22624 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:14.833930 22624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:14.834100 22624 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:14.836203 22624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "06e1e80f3bc844a798486387b8338204"
format_stamp: "Formatted at 2026-08-11 22:14:14 on dist-test-slave-vxj2"
server_key: "fc35b95cc2ec2ee2adeea8f07bc9ba8b"
server_key_iv: "a25ce6484207ea3d0b1f0d9e7e614f0d"
server_key_version: "encryptionkey@0"
I20260811 22:14:14.836751 22624 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "06e1e80f3bc844a798486387b8338204"
format_stamp: "Formatted at 2026-08-11 22:14:14 on dist-test-slave-vxj2"
server_key: "fc35b95cc2ec2ee2adeea8f07bc9ba8b"
server_key_iv: "a25ce6484207ea3d0b1f0d9e7e614f0d"
server_key_version: "encryptionkey@0"
I20260811 22:14:14.840804 22624 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:14:14.843979 22644 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:14.845553 22624 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:14.845702 22624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "06e1e80f3bc844a798486387b8338204"
format_stamp: "Formatted at 2026-08-11 22:14:14 on dist-test-slave-vxj2"
server_key: "fc35b95cc2ec2ee2adeea8f07bc9ba8b"
server_key_iv: "a25ce6484207ea3d0b1f0d9e7e614f0d"
server_key_version: "encryptionkey@0"
I20260811 22:14:14.845841 22624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:14.851234 22624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:14.854983 22624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:14.855216 22624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:14.863000 22624 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:34055
I20260811 22:14:14.863031 22706 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:34055 every 8 connection(s)
I20260811 22:14:14.864037 22624 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:14.867026 22707 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:14.872489 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 22624
I20260811 22:14:14.872664 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:14.872958 28862 external_mini_cluster.cc:1428] Setting key d61f9376e8c604c887c482da51e390a1
I20260811 22:14:14.873545 22707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204: Bootstrap starting.
I20260811 22:14:14.876153 22707 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:14.877141 22707 log.cc:824] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:14.878752 22710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:14.874595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55447 (local address 127.28.47.190:34055)
0811 22:14:14.875099 (+ 504us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:14.875114 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:14.875151 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:14:14.875848 (+ 697us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:14.875865 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:14.876157 (+ 292us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:14.876512 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:14.877449 (+ 937us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:14.877454 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:14.877466 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:14.877477 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:14.877535 (+ 58us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:14.877580 (+ 45us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:14.877708 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:14.877866 (+ 158us) server_negotiation.cc:299] Negotiation successful
0811 22:14:14.878037 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":170,"threads_started":1}
I20260811 22:14:14.879436 22707 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204: No bootstrap required, opened a new log
I20260811 22:14:14.882052 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:14.884171 22707 raft_consensus.cc:348] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e1e80f3bc844a798486387b8338204" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34055 } }
I20260811 22:14:14.884513 22707 raft_consensus.cc:374] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:14.884627 22707 raft_consensus.cc:729] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 06e1e80f3bc844a798486387b8338204, State: Initialized, Role: FOLLOWER
I20260811 22:14:14.885248 22707 consensus_queue.cc:260] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e1e80f3bc844a798486387b8338204" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34055 } }
I20260811 22:14:14.885447 22707 raft_consensus.cc:388] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:14.885528 22707 raft_consensus.cc:482] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:14.885655 22707 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:14.886077 22710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:14.884108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53821 (local address 127.28.47.190:34055)
0811 22:14:14.884240 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:14.884244 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:14.884545 (+ 301us) server_negotiation.cc:407] Connection header received
0811 22:14:14.884648 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:14.884651 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:14.884687 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:14.884842 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:14.885603 (+ 761us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:14.885609 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:14.885614 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:14.885631 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:14.885663 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:14.885676 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:14.885808 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:14.885903 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 22:14:14.885939 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260811 22:14:14.886886 22707 raft_consensus.cc:504] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e1e80f3bc844a798486387b8338204" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34055 } }
I20260811 22:14:14.887293 22707 leader_election.cc:302] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 06e1e80f3bc844a798486387b8338204; no voters:
I20260811 22:14:14.887598 22707 leader_election.cc:288] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:14.887737 22718 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:14.888005 22718 raft_consensus.cc:686] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [term 1 LEADER]: Becoming Leader. State: Replica: 06e1e80f3bc844a798486387b8338204, State: Running, Role: LEADER
I20260811 22:14:14.888383 22718 consensus_queue.cc:237] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e1e80f3bc844a798486387b8338204" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34055 } }
I20260811 22:14:14.888994 22707 sys_catalog.cc:564] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:14.890128 22720 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 06e1e80f3bc844a798486387b8338204. Latest consensus state: current_term: 1 leader_uuid: "06e1e80f3bc844a798486387b8338204" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e1e80f3bc844a798486387b8338204" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34055 } } }
I20260811 22:14:14.890293 22720 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:14.890252 22719 sys_catalog.cc:455] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "06e1e80f3bc844a798486387b8338204" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "06e1e80f3bc844a798486387b8338204" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34055 } } }
I20260811 22:14:14.890607 22719 sys_catalog.cc:458] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:14.890722 22727 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:14.893899 22727 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:14.899355 22727 catalog_manager.cc:1347] Generated new cluster ID: 3f845a93b2a34eac87078b8d26e9c0f8
I20260811 22:14:14.899444 22727 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:14.917235 22727 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:14.917670 22727 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:14.926745 22727 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 06e1e80f3bc844a798486387b8338204: Generated new TSK 0
I20260811 22:14:14.927500 22727 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:14.938665 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 22624
2026-08-11T22:14:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1547 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:15 dist-test-slave-vxj2 krb5kdc[22740](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:15 dist-test-slave-vxj2 krb5kdc[22740](info): set up 2 sockets
Aug 11 22:14:15 dist-test-slave-vxj2 krb5kdc[22740](info): commencing operation
krb5kdc: starting...
W20260811 22:14:16.080895 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.118s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:16 dist-test-slave-vxj2 krb5kdc[22740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:16Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:16Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:14:16.217721 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:39295
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:46467
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:39295
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:16.342235 22754 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:16.342597 22754 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:16.342692 22754 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:16.347062 22754 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:16.347168 22754 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:16.347206 22754 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:16.347256 22754 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:16.347298 22754 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:16.353089 22754 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:46467
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:39295
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:39295
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22754
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:16.354427 22754 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:16.355569 22754 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:16.364753 22762 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:16.364789 22764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:16.364916 22754 server_base.cc:1034] running on GCE node
W20260811 22:14:16.364771 22760 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:16.365796 22754 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:16.367101 22754 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:16.368335 22754 hybrid_clock.cc:648] HybridClock initialized: now 1786486456368303 us; error 47 us; skew 500 ppm
Aug 11 22:14:16 dist-test-slave-vxj2 krb5kdc[22740](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486456, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:16.371381 22754 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:14:16.372375 22754 webserver.cc:460] Webserver started at http://127.28.47.190:44851/ using document root <none> and password file <none>
I20260811 22:14:16.372946 22754 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:16.373010 22754 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:16.373261 22754 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:16.375561 22754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "7e23538eca9c4da7a82d8a7bc9b5a840"
format_stamp: "Formatted at 2026-08-11 22:14:16 on dist-test-slave-vxj2"
server_key: "de5a09877b9286fb8b946842d3c7096c"
server_key_iv: "91d1fb7401dcf6e39130fcdd0b92d4c8"
server_key_version: "encryptionkey@0"
I20260811 22:14:16.376163 22754 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "7e23538eca9c4da7a82d8a7bc9b5a840"
format_stamp: "Formatted at 2026-08-11 22:14:16 on dist-test-slave-vxj2"
server_key: "de5a09877b9286fb8b946842d3c7096c"
server_key_iv: "91d1fb7401dcf6e39130fcdd0b92d4c8"
server_key_version: "encryptionkey@0"
I20260811 22:14:16.380398 22754 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260811 22:14:16.383248 22771 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:16.384826 22754 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:14:16.384956 22754 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "7e23538eca9c4da7a82d8a7bc9b5a840"
format_stamp: "Formatted at 2026-08-11 22:14:16 on dist-test-slave-vxj2"
server_key: "de5a09877b9286fb8b946842d3c7096c"
server_key_iv: "91d1fb7401dcf6e39130fcdd0b92d4c8"
server_key_version: "encryptionkey@0"
I20260811 22:14:16.385061 22754 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:16.398309 22754 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:16.401798 22754 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:16.402019 22754 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:16.409658 22754 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:39295
I20260811 22:14:16.409689 22832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:39295 every 8 connection(s)
I20260811 22:14:16.410713 22754 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:16.414027 22835 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:16.416121 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 22754
I20260811 22:14:16.416272 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:16.416561 28862 external_mini_cluster.cc:1428] Setting key f47023ad51b8acd1a1be4268f9ed2346
I20260811 22:14:16.421121 22835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840: Bootstrap starting.
I20260811 22:14:16.423568 22835 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:16.424466 22835 log.cc:824] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:14:16 dist-test-slave-vxj2 krb5kdc[22740](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486456, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:14:16.427065 22835 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840: No bootstrap required, opened a new log
I20260811 22:14:16.430850 22835 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e23538eca9c4da7a82d8a7bc9b5a840" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 39295 } }
I20260811 22:14:16.431173 22835 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:16.431255 22835 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7e23538eca9c4da7a82d8a7bc9b5a840, State: Initialized, Role: FOLLOWER
I20260811 22:14:16.431737 22835 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e23538eca9c4da7a82d8a7bc9b5a840" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 39295 } }
I20260811 22:14:16.431864 22835 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:16.431998 22835 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:16.432114 22835 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:16.432240 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:16.418158 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51983 (local address 127.28.47.190:39295)
0811 22:14:16.418626 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:16.418642 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:16.418679 (+ 37us) server_negotiation.cc:407] Connection header received
0811 22:14:16.419382 (+ 703us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:16.419401 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:16.419771 (+ 370us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:16.420112 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:16.420960 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:16.422055 (+ 1095us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:16.422949 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:16.423436 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:16.423611 (+ 175us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:16.426403 (+ 2792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:16.426409 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:16.426424 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:16.426469 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:16.428567 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:16.429095 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:16.429101 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:16.429106 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:16.429307 (+ 201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:16.429635 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:16.429640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:16.429642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:16.430116 (+ 474us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:16.430360 (+ 244us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:16.430683 (+ 323us) server_negotiation.cc:299] Negotiation successful
0811 22:14:16.431059 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":149,"threads_started":1}
I20260811 22:14:16.433208 22835 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e23538eca9c4da7a82d8a7bc9b5a840" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 39295 } }
I20260811 22:14:16.433578 22835 leader_election.cc:302] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7e23538eca9c4da7a82d8a7bc9b5a840; no voters:
I20260811 22:14:16.433905 22835 leader_election.cc:288] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:16.434036 22841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:16.434269 22841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [term 1 LEADER]: Becoming Leader. State: Replica: 7e23538eca9c4da7a82d8a7bc9b5a840, State: Running, Role: LEADER
I20260811 22:14:16.434973 22841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e23538eca9c4da7a82d8a7bc9b5a840" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 39295 } }
I20260811 22:14:16.435544 22835 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:16.435986 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:16.437032 22842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7e23538eca9c4da7a82d8a7bc9b5a840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e23538eca9c4da7a82d8a7bc9b5a840" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 39295 } } }
I20260811 22:14:16.437295 22842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:16.437640 22842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7e23538eca9c4da7a82d8a7bc9b5a840. Latest consensus state: current_term: 1 leader_uuid: "7e23538eca9c4da7a82d8a7bc9b5a840" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7e23538eca9c4da7a82d8a7bc9b5a840" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 39295 } } }
I20260811 22:14:16.437837 22842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:16.438683 22853 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:16.442852 22853 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:16.447970 22838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:16.438645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42793 (local address 127.28.47.190:39295)
0811 22:14:16.438857 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:16.438864 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:16.439410 (+ 546us) server_negotiation.cc:407] Connection header received
0811 22:14:16.439596 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:16.439599 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:16.439655 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:16.439972 (+ 317us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:16.440827 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:16.441452 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:16.442310 (+ 858us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:16.442551 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:16.442665 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:16.444102 (+ 1437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:16.444106 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:16.444109 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:16.444138 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:16.445846 (+ 1708us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:16.446427 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:16.446431 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:16.446432 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:16.446540 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:16.446938 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:16.446941 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:16.446943 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:16.447176 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:16.447311 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:16.447642 (+ 331us) server_negotiation.cc:299] Negotiation successful
0811 22:14:16.447792 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":118}
I20260811 22:14:16.449795 22853 catalog_manager.cc:1347] Generated new cluster ID: cf679ce54078443db192eca02f86f9df
I20260811 22:14:16.449920 22853 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:16.464176 22853 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:16.465426 22853 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:16.484436 22853 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7e23538eca9c4da7a82d8a7bc9b5a840: Generated new TSK 0
I20260811 22:14:16.485339 22853 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:16.498095 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 22754
2026-08-11T22:14:16Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1560 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:16 dist-test-slave-vxj2 krb5kdc[22868](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:16 dist-test-slave-vxj2 krb5kdc[22868](info): set up 2 sockets
Aug 11 22:14:16 dist-test-slave-vxj2 krb5kdc[22868](info): commencing operation
krb5kdc: starting...
W20260811 22:14:17.658074 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:17 dist-test-slave-vxj2 krb5kdc[22868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486457, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:14:17.796283 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:36607
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:43157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:36607
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:17.912004 22882 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:17.912364 22882 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:17.912444 22882 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:17.916788 22882 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:17.916889 22882 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:17.916925 22882 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:17.916965 22882 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:17.917006 22882 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:17.922689 22882 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:43157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:36607
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:36607
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22882
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:17.923925 22882 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:17.925009 22882 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:17.934223 22888 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:17.934234 22889 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:17.934369 22882 server_base.cc:1034] running on GCE node
W20260811 22:14:17.934227 22891 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:17.935122 22882 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:17.936398 22882 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:17.937604 22882 hybrid_clock.cc:648] HybridClock initialized: now 1786486457937586 us; error 29 us; skew 500 ppm
Aug 11 22:14:17 dist-test-slave-vxj2 krb5kdc[22868](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486457, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:17.940697 22882 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:14:17.941742 22882 webserver.cc:460] Webserver started at http://127.28.47.190:37361/ using document root <none> and password file <none>
I20260811 22:14:17.942322 22882 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:17.942394 22882 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:17.942572 22882 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:17.944730 22882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "1ef95cecdf78468fbd4fd501a674f22b"
format_stamp: "Formatted at 2026-08-11 22:14:17 on dist-test-slave-vxj2"
server_key: "59c67131d7aa2d2fac8c8e1551730670"
server_key_iv: "70f3b27c3f13f94633021a6cc84c26c6"
server_key_version: "encryptionkey@0"
I20260811 22:14:17.945359 22882 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "1ef95cecdf78468fbd4fd501a674f22b"
format_stamp: "Formatted at 2026-08-11 22:14:17 on dist-test-slave-vxj2"
server_key: "59c67131d7aa2d2fac8c8e1551730670"
server_key_iv: "70f3b27c3f13f94633021a6cc84c26c6"
server_key_version: "encryptionkey@0"
I20260811 22:14:17.949554 22882 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260811 22:14:17.952265 22900 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:17.953816 22882 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:17.953989 22882 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "1ef95cecdf78468fbd4fd501a674f22b"
format_stamp: "Formatted at 2026-08-11 22:14:17 on dist-test-slave-vxj2"
server_key: "59c67131d7aa2d2fac8c8e1551730670"
server_key_iv: "70f3b27c3f13f94633021a6cc84c26c6"
server_key_version: "encryptionkey@0"
I20260811 22:14:17.954123 22882 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:17.974083 22882 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:17.977619 22882 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:17.977881 22882 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:17.986063 22882 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:36607
I20260811 22:14:17.986099 22959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:36607 every 8 connection(s)
I20260811 22:14:17.987178 22882 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:17.990435 22960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:17.994495 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 22882
I20260811 22:14:17.994658 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:17.995002 28862 external_mini_cluster.cc:1428] Setting key 73ec5b1bfd80070586a6a43f7b592c5a
I20260811 22:14:17.997335 22960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b: Bootstrap starting.
I20260811 22:14:17.999866 22960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:18.000785 22960 log.cc:824] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:18.003660 22960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b: No bootstrap required, opened a new log
I20260811 22:14:18.005770 22963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:17.996713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49131 (local address 127.28.47.190:36607)
0811 22:14:17.997297 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:17.997315 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:17.997357 (+ 42us) server_negotiation.cc:407] Connection header received
0811 22:14:17.998098 (+ 741us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:17.998116 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:17.998494 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:17.998860 (+ 366us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:17.999347 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:18.000453 (+ 1106us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:18.001610 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:18.002376 (+ 766us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:18.002561 (+ 185us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:18.003465 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:18.003472 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:18.003491 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:18.004061 (+ 570us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:18.004144 (+ 83us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:18.004178 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:18.004286 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:18.004476 (+ 190us) server_negotiation.cc:299] Negotiation successful
0811 22:14:18.004708 (+ 232us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":155,"threads_started":1}
I20260811 22:14:18.008638 22960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef95cecdf78468fbd4fd501a674f22b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36607 } }
I20260811 22:14:18.008888 22960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:18.008925 22960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1ef95cecdf78468fbd4fd501a674f22b, State: Initialized, Role: FOLLOWER
I20260811 22:14:18.009330 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:18.009439 22960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef95cecdf78468fbd4fd501a674f22b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36607 } }
I20260811 22:14:18.009605 22960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:18.009692 22960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:18.009812 22960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:18.010923 22960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef95cecdf78468fbd4fd501a674f22b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36607 } }
I20260811 22:14:18.011312 22960 leader_election.cc:302] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1ef95cecdf78468fbd4fd501a674f22b; no voters:
I20260811 22:14:18.011744 22960 leader_election.cc:288] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:18.011936 22972 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:18.012166 22972 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [term 1 LEADER]: Becoming Leader. State: Replica: 1ef95cecdf78468fbd4fd501a674f22b, State: Running, Role: LEADER
I20260811 22:14:18.012881 22972 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef95cecdf78468fbd4fd501a674f22b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36607 } }
I20260811 22:14:18.013348 22960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:18.017017 22977 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1ef95cecdf78468fbd4fd501a674f22b. Latest consensus state: current_term: 1 leader_uuid: "1ef95cecdf78468fbd4fd501a674f22b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef95cecdf78468fbd4fd501a674f22b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36607 } } }
I20260811 22:14:18.017059 22975 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1ef95cecdf78468fbd4fd501a674f22b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1ef95cecdf78468fbd4fd501a674f22b" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 36607 } } }
I20260811 22:14:18.017198 22977 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:18.017215 22975 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:18.017771 22963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:18.011657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50097 (local address 127.28.47.190:36607)
0811 22:14:18.011779 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:18.011784 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:18.012499 (+ 715us) server_negotiation.cc:407] Connection header received
0811 22:14:18.012591 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:18.012593 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:18.012646 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:18.012738 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:18.013293 (+ 555us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:18.014087 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:18.015377 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:18.016193 (+ 816us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:18.016303 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:18.017312 (+ 1009us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:18.017317 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:18.017320 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:18.017334 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:18.017362 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:18.017374 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:18.017470 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:18.017565 (+ 95us) server_negotiation.cc:299] Negotiation successful
0811 22:14:18.017622 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
W20260811 22:14:18.019763 22991 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:14:18.019891 22991 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:14:18.020002 22993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:18.020807 22993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:18.027001 22993 catalog_manager.cc:1347] Generated new cluster ID: 4b1ecd3a29c04b4e934aeb169582c400
I20260811 22:14:18.027091 22993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:18.043763 22993 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:18.044819 22993 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:18.056538 22993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1ef95cecdf78468fbd4fd501a674f22b: Generated new TSK 0
I20260811 22:14:18.057482 22993 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:18.063385 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 22882
2026-08-11T22:14:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:18 dist-test-slave-vxj2 krb5kdc[22998](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:18 dist-test-slave-vxj2 krb5kdc[22998](info): set up 2 sockets
Aug 11 22:14:18 dist-test-slave-vxj2 krb5kdc[22998](info): commencing operation
krb5kdc: starting...
W20260811 22:14:19.280344 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.188s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:19 dist-test-slave-vxj2 krb5kdc[22998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486459, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:14:19.421552 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:42589
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:43767
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:42589
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:19.547142 23014 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:19.547453 23014 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:19.547500 23014 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:19.551754 23014 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:19.551841 23014 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:19.551867 23014 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:19.551884 23014 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:19.551929 23014 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:19.557581 23014 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:43767
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:42589
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:42589
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23014
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:19.558779 23014 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:19.559916 23014 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:19.569128 23023 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:19.569073 23020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:19.569069 23021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:19.569380 23014 server_base.cc:1034] running on GCE node
I20260811 22:14:19.570081 23014 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:19.571290 23014 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:19.572500 23014 hybrid_clock.cc:648] HybridClock initialized: now 1786486459572481 us; error 34 us; skew 500 ppm
Aug 11 22:14:19 dist-test-slave-vxj2 krb5kdc[22998](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486459, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:19.580971 23014 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:14:19.582229 23014 webserver.cc:460] Webserver started at http://127.28.47.190:32875/ using document root <none> and password file <none>
I20260811 22:14:19.582899 23014 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:19.583000 23014 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:19.583248 23014 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:19.585618 23014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "480cec8aa02c4290b69aa7217e001f29"
format_stamp: "Formatted at 2026-08-11 22:14:19 on dist-test-slave-vxj2"
server_key: "b347045ca31ddaaa94165eca81c65508"
server_key_iv: "2a7d3ecb2c8f7b13c65cd7058903dd2c"
server_key_version: "encryptionkey@0"
I20260811 22:14:19.586220 23014 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "480cec8aa02c4290b69aa7217e001f29"
format_stamp: "Formatted at 2026-08-11 22:14:19 on dist-test-slave-vxj2"
server_key: "b347045ca31ddaaa94165eca81c65508"
server_key_iv: "2a7d3ecb2c8f7b13c65cd7058903dd2c"
server_key_version: "encryptionkey@0"
I20260811 22:14:19.590358 23014 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:14:19.593048 23030 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:19.594513 23014 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260811 22:14:19.594688 23014 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "480cec8aa02c4290b69aa7217e001f29"
format_stamp: "Formatted at 2026-08-11 22:14:19 on dist-test-slave-vxj2"
server_key: "b347045ca31ddaaa94165eca81c65508"
server_key_iv: "2a7d3ecb2c8f7b13c65cd7058903dd2c"
server_key_version: "encryptionkey@0"
I20260811 22:14:19.594830 23014 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:19.599941 23014 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:19.603586 23014 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:19.603821 23014 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:19.611423 23014 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:42589
I20260811 22:14:19.611444 23087 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:42589 every 8 connection(s)
I20260811 22:14:19.612445 23014 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:19.615687 23088 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:19.619958 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 23014
I20260811 22:14:19.620133 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:19.620478 28862 external_mini_cluster.cc:1428] Setting key 996d2e768937f080be3c74e0abec7f22
I20260811 22:14:19.623078 23088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29: Bootstrap starting.
I20260811 22:14:19.625834 23088 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:19.626829 23088 log.cc:824] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:19.627184 23092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:19.622448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49819 (local address 127.28.47.190:42589)
0811 22:14:19.623238 (+ 790us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:19.623268 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:19.623323 (+ 55us) server_negotiation.cc:407] Connection header received
0811 22:14:19.624110 (+ 787us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:19.624131 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:19.624451 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:19.624852 (+ 401us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:19.625798 (+ 946us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:19.625810 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:19.625822 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:19.625835 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:19.625919 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:19.625954 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:19.626073 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:19.626197 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 22:14:19.626367 (+ 170us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":551,"thread_start_us":173,"threads_started":1}
I20260811 22:14:19.629295 23088 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29: No bootstrap required, opened a new log
I20260811 22:14:19.630611 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:19.635176 23092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:19.633144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50319 (local address 127.28.47.190:42589)
0811 22:14:19.633298 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:19.633303 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:19.633539 (+ 236us) server_negotiation.cc:407] Connection header received
0811 22:14:19.633715 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:19.633717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:19.633790 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:19.633856 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:19.634692 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:19.634695 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:19.634698 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0811 22:14:19.634709 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:19.634745 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0811 22:14:19.634755 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:19.634836 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:19.634960 (+ 124us) server_negotiation.cc:299] Negotiation successful
0811 22:14:19.634995 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260811 22:14:19.635113 23088 raft_consensus.cc:348] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480cec8aa02c4290b69aa7217e001f29" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42589 } }
I20260811 22:14:19.635474 23088 raft_consensus.cc:374] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260811 22:14:19.635548 23088 raft_consensus.cc:729] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 480cec8aa02c4290b69aa7217e001f29, State: Initialized, Role: FOLLOWER
I20260811 22:14:19.636116 23088 consensus_queue.cc:260] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480cec8aa02c4290b69aa7217e001f29" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42589 } }
I20260811 22:14:19.636313 23088 raft_consensus.cc:388] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260811 22:14:19.636408 23088 raft_consensus.cc:482] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260811 22:14:19.636530 23088 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:19.637748 23088 raft_consensus.cc:504] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480cec8aa02c4290b69aa7217e001f29" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42589 } }
I20260811 22:14:19.638206 23088 leader_election.cc:302] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 480cec8aa02c4290b69aa7217e001f29; no voters:
I20260811 22:14:19.638592 23088 leader_election.cc:288] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260811 22:14:19.638744 23101 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:19.639020 23101 raft_consensus.cc:686] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [term 1 LEADER]: Becoming Leader. State: Replica: 480cec8aa02c4290b69aa7217e001f29, State: Running, Role: LEADER
I20260811 22:14:19.639422 23101 consensus_queue.cc:237] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480cec8aa02c4290b69aa7217e001f29" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42589 } }
I20260811 22:14:19.640084 23088 sys_catalog.cc:564] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:19.641496 23103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 480cec8aa02c4290b69aa7217e001f29. Latest consensus state: current_term: 1 leader_uuid: "480cec8aa02c4290b69aa7217e001f29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480cec8aa02c4290b69aa7217e001f29" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42589 } } }
I20260811 22:14:19.641942 23103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:19.641824 23102 sys_catalog.cc:455] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "480cec8aa02c4290b69aa7217e001f29" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "480cec8aa02c4290b69aa7217e001f29" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 42589 } } }
I20260811 22:14:19.642194 23102 sys_catalog.cc:458] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29 [sys.catalog]: This master's current role is: LEADER
W20260811 22:14:19.645666 23120 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:14:19.645835 23120 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:14:19.645927 23115 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:19.646919 23115 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:19.653129 23115 catalog_manager.cc:1347] Generated new cluster ID: a7e656db633645bebebdb871f6efbcc3
I20260811 22:14:19.653271 23115 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:19.666710 23115 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:19.667101 23115 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:19.675213 23115 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 480cec8aa02c4290b69aa7217e001f29: Generated new TSK 0
I20260811 22:14:19.676074 23115 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260811 22:14:19.681849 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 23014
2026-08-11T22:14:19Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1614 ms)
[----------] 18 tests from AuthTokenIssuingTest (28422 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:19 dist-test-slave-vxj2 krb5kdc[23126](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:19 dist-test-slave-vxj2 krb5kdc[23126](info): set up 2 sockets
Aug 11 22:14:19 dist-test-slave-vxj2 krb5kdc[23126](info): commencing operation
krb5kdc: starting...
W20260811 22:14:20.832284 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.125s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:20 dist-test-slave-vxj2 krb5kdc[23126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:14:20.973611 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45825
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:46093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:45825,127.28.47.189:37013,127.28.47.188:33873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:21.090366 23140 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:14:21.090746 23140 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:21.090830 23140 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:21.090917 23140 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:21.095149 23140 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:21.095253 23140 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:21.095294 23140 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:21.095333 23140 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:21.095368 23140 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:21.100867 23140 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:46093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:45825,127.28.47.189:37013,127.28.47.188:33873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:45825
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23140
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:21.102120 23140 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:21.103225 23140 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:21.112304 23150 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:21.112304 23147 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:21.112381 23146 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:21.112695 23140 server_base.cc:1034] running on GCE node
I20260811 22:14:21.113387 23140 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:21.114576 23140 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:21.115805 23140 hybrid_clock.cc:648] HybridClock initialized: now 1786486461115774 us; error 43 us; skew 500 ppm
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486461, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:21.118860 23140 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:14:21.119799 23140 webserver.cc:460] Webserver started at http://127.28.47.190:39259/ using document root <none> and password file <none>
I20260811 22:14:21.120363 23140 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:21.120426 23140 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:21.120770 23140 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:21.123246 23140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "193a033c826d43faaaec04daedd80477"
format_stamp: "Formatted at 2026-08-11 22:14:21 on dist-test-slave-vxj2"
server_key: "1170287eba1e7e13fb081138f87aa441"
server_key_iv: "9da8f4d8590d0831bd39794592fb33d3"
server_key_version: "encryptionkey@0"
I20260811 22:14:21.123845 23140 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "193a033c826d43faaaec04daedd80477"
format_stamp: "Formatted at 2026-08-11 22:14:21 on dist-test-slave-vxj2"
server_key: "1170287eba1e7e13fb081138f87aa441"
server_key_iv: "9da8f4d8590d0831bd39794592fb33d3"
server_key_version: "encryptionkey@0"
I20260811 22:14:21.128198 23140 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:14:21.131188 23157 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:21.132623 23140 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:14:21.132813 23140 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "193a033c826d43faaaec04daedd80477"
format_stamp: "Formatted at 2026-08-11 22:14:21 on dist-test-slave-vxj2"
server_key: "1170287eba1e7e13fb081138f87aa441"
server_key_iv: "9da8f4d8590d0831bd39794592fb33d3"
server_key_version: "encryptionkey@0"
I20260811 22:14:21.132988 23140 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:21.147639 23140 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:21.150918 23140 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:21.151153 23140 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:21.159037 23140 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:45825
I20260811 22:14:21.159361 23213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:45825 every 8 connection(s)
I20260811 22:14:21.160220 23140 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:21.161502 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 23140
I20260811 22:14:21.161652 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:21.161980 28862 external_mini_cluster.cc:1428] Setting key 3b5a025490345439d1223b12d2508e6b
I20260811 22:14:21.164183 23214 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:21.168864 23214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.47.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.189@KRBTEST.COM" created.
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486461, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:14:21.187628 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.171581 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45825 (local address 127.0.0.1:35722)
0811 22:14:21.172133 (+ 552us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.172186 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:21.173282 (+ 1096us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:21.174247 (+ 965us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:21.174256 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:21.174886 (+ 630us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:21.175324 (+ 438us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.175349 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.176644 (+ 1295us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.176647 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.177533 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.177544 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.178496 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.178503 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.178670 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.179343 (+ 673us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:21.179371 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:21.181553 (+ 2182us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:21.184158 (+ 2605us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.184168 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.184178 (+ 10us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.184459 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.184941 (+ 482us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.184943 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.184945 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.185075 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.185964 (+ 889us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:21.185970 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:21.186113 (+ 143us) client_negotiation.cc:769] Sending connection context
0811 22:14:21.186263 (+ 150us) client_negotiation.cc:240] Negotiation successful
0811 22:14:21.186766 (+ 503us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":111,"thread_start_us":161,"threads_started":1}
I20260811 22:14:21.187631 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.171581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35722 (local address 127.28.47.190:45825)
0811 22:14:21.172141 (+ 560us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.172157 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.172305 (+ 148us) server_negotiation.cc:407] Connection header received
0811 22:14:21.173666 (+ 1361us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.173675 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.173974 (+ 299us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.174158 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.175522 (+ 1364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.176493 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.177761 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.178297 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.178446 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.181748 (+ 3302us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.181758 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.181777 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.181807 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.183993 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.184624 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.184635 (+ 11us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.184638 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.184796 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.185287 (+ 491us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.185290 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.185291 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.185826 (+ 535us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.185960 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.186384 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.186680 (+ 296us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":35,"server-negotiator.queue_time_us":417,"spinlock_wait_cycles":29056,"thread_start_us":251,"threads_started":1}
I20260811 22:14:21.190305 23214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:14:21.191185 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.190904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:37013 (local address 127.0.0.1:32916)
0811 22:14:21.191044 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.191121 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:37013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 22:14:21.191437 23158 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.47.189:37013: connect: Connection refused (error 111)
I20260811 22:14:21.192417 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.192191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:37013 (local address 127.0.0.1:32920)
0811 22:14:21.192301 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.192349 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:37013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 22:14:21.192690 23214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.189:37013: Network error: Client connection negotiation failed: client connection to 127.28.47.189:37013: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.189.keytab.
Entry for principal kudu/127.28.47.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.189.keytab.
I20260811 22:14:21.196393 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.189:37013
--webserver_interface=127.28.47.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.189.keytab
--principal=kudu/127.28.47.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:46093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:45825,127.28.47.189:37013,127.28.47.188:33873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:14:21.241957 23214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } attempt: 1
I20260811 22:14:21.242920 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.242594 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:37013 (local address 127.0.0.1:32930)
0811 22:14:21.242772 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.242839 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:37013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260811 22:14:21.243902 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.243670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:37013 (local address 127.0.0.1:32944)
0811 22:14:21.243787 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.243838 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:37013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
W20260811 22:14:21.244155 23214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.189:37013: Network error: Client connection negotiation failed: client connection to 127.28.47.189:37013: connect: Connection refused (error 111)
I20260811 22:14:21.312345 23214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } attempt: 2
I20260811 22:14:21.313427 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.313076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:37013 (local address 127.0.0.1:32954)
0811 22:14:21.313256 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.313345 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:37013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":55}
I20260811 22:14:21.314335 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.314092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:37013 (local address 127.0.0.1:32968)
0811 22:14:21.314215 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.314264 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:37013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 22:14:21.314590 23214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.189:37013: Network error: Client connection negotiation failed: client connection to 127.28.47.189:37013: connect: Connection refused (error 111)
W20260811 22:14:21.325995 23221 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:14:21.326331 23221 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:21.326397 23221 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:21.326498 23221 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:21.330775 23221 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:21.330860 23221 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:21.330881 23221 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:21.330899 23221 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:21.330914 23221 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:21.336870 23221 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:46093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:45825,127.28.47.189:37013,127.28.47.188:33873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.189.keytab
--principal=kudu/127.28.47.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.189:37013
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.47.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23221
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:21.338176 23221 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:21.339286 23221 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:21.348652 23229 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:21.348658 23227 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:21.348660 23226 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:21.348701 23221 server_base.cc:1034] running on GCE node
I20260811 22:14:21.349577 23221 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:21.350764 23221 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:21.351940 23221 hybrid_clock.cc:648] HybridClock initialized: now 1786486461351925 us; error 31 us; skew 500 ppm
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486461, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:21.354944 23221 init.cc:377] Logged in from keytab as kudu/127.28.47.189@KRBTEST.COM (short username kudu)
I20260811 22:14:21.355947 23221 webserver.cc:460] Webserver started at http://127.28.47.189:40295/ using document root <none> and password file <none>
I20260811 22:14:21.356549 23221 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:21.356612 23221 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:21.356841 23221 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:21.359041 23221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/data/instance:
uuid: "f6c0f06b73604170bbd8cd3d75c5aca6"
format_stamp: "Formatted at 2026-08-11 22:14:21 on dist-test-slave-vxj2"
server_key: "54815020d5f9a2858f9ac7f5967e85b6"
server_key_iv: "bf4f895fecf5d360d4f26b9daa0636df"
server_key_version: "encryptionkey@0"
I20260811 22:14:21.359651 23221 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/wal/instance:
uuid: "f6c0f06b73604170bbd8cd3d75c5aca6"
format_stamp: "Formatted at 2026-08-11 22:14:21 on dist-test-slave-vxj2"
server_key: "54815020d5f9a2858f9ac7f5967e85b6"
server_key_iv: "bf4f895fecf5d360d4f26b9daa0636df"
server_key_version: "encryptionkey@0"
I20260811 22:14:21.363785 23221 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260811 22:14:21.366441 23236 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:21.367898 23221 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260811 22:14:21.368063 23221 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/wal
uuid: "f6c0f06b73604170bbd8cd3d75c5aca6"
format_stamp: "Formatted at 2026-08-11 22:14:21 on dist-test-slave-vxj2"
server_key: "54815020d5f9a2858f9ac7f5967e85b6"
server_key_iv: "bf4f895fecf5d360d4f26b9daa0636df"
server_key_version: "encryptionkey@0"
I20260811 22:14:21.368187 23221 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:21.400452 23221 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:21.404440 23221 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:21.404716 23221 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:21.405804 23214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } attempt: 3
I20260811 22:14:21.406787 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.406430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:37013 (local address 127.0.0.1:32976)
0811 22:14:21.406626 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.406703 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:37013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260811 22:14:21.407697 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.407460 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:37013 (local address 127.0.0.1:32982)
0811 22:14:21.407581 (+ 121us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.407638 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:37013: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:14:21.407835 23214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.189:37013: Network error: Client connection negotiation failed: client connection to 127.28.47.189:37013: connect: Connection refused (error 111)
I20260811 22:14:21.413429 23221 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.189:37013
I20260811 22:14:21.413729 23304 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.189:37013 every 8 connection(s)
I20260811 22:14:21.414604 23221 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/data/info.pb
I20260811 22:14:21.415136 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 23221
I20260811 22:14:21.415288 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-1/wal/instance
I20260811 22:14:21.415570 28862 external_mini_cluster.cc:1428] Setting key 7eab7a0affd388afa5b0eddfbc54af9c
I20260811 22:14:21.418298 23305 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:21.422602 23305 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.47.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.188@KRBTEST.COM" created.
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486461, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.189@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:14:21.439550 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.424743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35734 (local address 127.28.47.190:45825)
0811 22:14:21.425002 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.425071 (+ 69us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.426165 (+ 1094us) server_negotiation.cc:407] Connection header received
0811 22:14:21.427442 (+ 1277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.427446 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.427519 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.427602 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.429059 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.429789 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.431353 (+ 1564us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.431617 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.431799 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.434940 (+ 3141us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.434944 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.434946 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.434975 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.436665 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.437443 (+ 778us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.437446 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.437447 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.437531 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.437922 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.437925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.437926 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.438217 (+ 291us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.438303 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.439103 (+ 800us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.439295 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260811 22:14:21.440392 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.425251 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45825 (local address 127.0.0.1:35734)
0811 22:14:21.425990 (+ 739us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.426044 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:21.427096 (+ 1052us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:21.427848 (+ 752us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:21.427856 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:21.428234 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:21.428783 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.428799 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.429942 (+ 1143us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.429947 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.431196 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.431208 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.431854 (+ 646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.431858 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.432060 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.432752 (+ 692us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:21.432775 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:21.434704 (+ 1929us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:21.436805 (+ 2101us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.436815 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.436837 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.437347 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.437624 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.437627 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.437628 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.437816 (+ 188us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.438331 (+ 515us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:21.438338 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:21.438652 (+ 314us) client_negotiation.cc:769] Sending connection context
0811 22:14:21.438925 (+ 273us) client_negotiation.cc:240] Negotiation successful
0811 22:14:21.439236 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":407,"thread_start_us":175,"threads_started":1}
I20260811 22:14:21.441987 23305 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.188.keytab.
Entry for principal kudu/127.28.47.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.188.keytab.
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486461, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.189@KRBTEST.COM for kudu/127.28.47.189@KRBTEST.COM
I20260811 22:14:21.448154 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.188:33873
--webserver_interface=127.28.47.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.188.keytab
--principal=kudu/127.28.47.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:46093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:45825,127.28.47.189:37013,127.28.47.188:33873
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:14:21.452291 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.442519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:37013 (local address 127.0.0.1:32998)
0811 22:14:21.442643 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.442659 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:21.442737 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:21.443585 (+ 848us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:21.443591 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:21.443788 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:21.443888 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.443894 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.445050 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.445053 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.445724 (+ 671us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.445734 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.446493 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.446496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.446608 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.447129 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:21.447141 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:21.448555 (+ 1414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:21.450632 (+ 2077us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.450637 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.450639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.450839 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.451139 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.451144 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.451146 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.451229 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.451724 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:21.451727 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:21.451819 (+ 92us) client_negotiation.cc:769] Sending connection context
0811 22:14:21.451936 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:14:21.452079 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50,"mutex_wait_us":28}
I20260811 22:14:21.452306 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.442699 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32998 (local address 127.28.47.189:37013)
0811 22:14:21.443139 (+ 440us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.443149 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.443171 (+ 22us) server_negotiation.cc:407] Connection header received
0811 22:14:21.443236 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.443240 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.443400 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.443582 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.444019 (+ 437us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.444940 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.445849 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.446370 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.446482 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.448697 (+ 2215us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.448715 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.448733 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.448753 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.450498 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.450955 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.450962 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.450963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.451045 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.451332 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.451335 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.451336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.451605 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.451752 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.452036 (+ 284us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.452168 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":349,"thread_start_us":139,"threads_started":1}
I20260811 22:14:21.454789 23305 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:14:21.455657 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.455369 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:33873 (local address 127.0.0.1:51004)
0811 22:14:21.455506 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.455589 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 22:14:21.455948 23241 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
I20260811 22:14:21.456820 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.456603 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:33873 (local address 127.0.0.1:51016)
0811 22:14:21.456712 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.456766 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260811 22:14:21.456995 23305 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:33873: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
I20260811 22:14:21.506330 23305 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } attempt: 1
I20260811 22:14:21.507474 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.507067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:33873 (local address 127.0.0.1:51026)
0811 22:14:21.507285 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.507356 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":96}
I20260811 22:14:21.508404 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.508157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:33873 (local address 127.0.0.1:51038)
0811 22:14:21.508279 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.508338 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:14:21.508548 23305 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:33873: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
I20260811 22:14:21.551049 23214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } attempt: 4
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486461, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for kudu/127.28.47.189@KRBTEST.COM
I20260811 22:14:21.561194 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.551867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33010 (local address 127.28.47.189:37013)
0811 22:14:21.552018 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.552023 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.552046 (+ 23us) server_negotiation.cc:407] Connection header received
0811 22:14:21.552188 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.552192 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.552264 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.552372 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.552815 (+ 443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.553666 (+ 851us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.554473 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.555549 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.555659 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.557944 (+ 2285us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.557948 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.557950 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.557974 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.559455 (+ 1481us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.559948 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.559950 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.559952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.560029 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.560329 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.560331 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.560332 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.560531 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.560608 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.560925 (+ 317us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.561029 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260811 22:14:21.561228 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.551757 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:37013 (local address 127.0.0.1:33010)
0811 22:14:21.551958 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.551977 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:21.552077 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:21.552403 (+ 326us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:21.552406 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:21.552583 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:21.552684 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.552689 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.553801 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.553805 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.554354 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.554365 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.555685 (+ 1320us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.555687 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.555784 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.556347 (+ 563us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:21.556360 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:21.557794 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:21.559573 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.559577 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.559579 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.559844 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.560122 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.560125 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.560127 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.560228 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.560632 (+ 404us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:21.560635 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:21.560747 (+ 112us) client_negotiation.cc:769] Sending connection context
0811 22:14:21.560865 (+ 118us) client_negotiation.cc:240] Negotiation successful
0811 22:14:21.561019 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 22:14:21.562315 23214 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:14:21.563138 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.562853 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:33873 (local address 127.0.0.1:51044)
0811 22:14:21.562992 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.563058 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260811 22:14:21.564405 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.564122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:33873 (local address 127.0.0.1:51048)
0811 22:14:21.564255 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.564311 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:14:21.564535 23214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:33873: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
W20260811 22:14:21.573315 23311 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:14:21.573666 23311 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:21.573729 23311 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:21.573837 23311 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260811 22:14:21.576716 23305 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } attempt: 2
I20260811 22:14:21.577569 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.577279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:33873 (local address 127.0.0.1:51052)
0811 22:14:21.577428 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.577491 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:14:21.578265 23311 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
I20260811 22:14:21.578270 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.578062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:33873 (local address 127.0.0.1:51062)
0811 22:14:21.578170 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.578221 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
W20260811 22:14:21.578341 23311 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:21.578382 23311 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:21.578419 23311 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:21.578455 23311 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260811 22:14:21.578467 23305 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:33873: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
I20260811 22:14:21.583961 23311 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:46093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:45825,127.28.47.189:37013,127.28.47.188:33873
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.188.keytab
--principal=kudu/127.28.47.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.188:33873
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.47.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23311
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:21.585093 23311 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:21.586196 23311 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:21.595407 23318 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:21.595504 23311 server_base.cc:1034] running on GCE node
W20260811 22:14:21.595417 23319 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:21.595417 23321 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:21.596576 23311 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:21.597904 23311 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:21.599099 23311 hybrid_clock.cc:648] HybridClock initialized: now 1786486461599060 us; error 50 us; skew 500 ppm
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486461, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:21.602373 23311 init.cc:377] Logged in from keytab as kudu/127.28.47.188@KRBTEST.COM (short username kudu)
I20260811 22:14:21.603430 23311 webserver.cc:460] Webserver started at http://127.28.47.188:46779/ using document root <none> and password file <none>
I20260811 22:14:21.604043 23311 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:21.604115 23311 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:21.604364 23311 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:21.606909 23311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/data/instance:
uuid: "1fcb604df58a41fb8759726b378900c2"
format_stamp: "Formatted at 2026-08-11 22:14:21 on dist-test-slave-vxj2"
server_key: "a2b712cad15c4d48526225a113da5577"
server_key_iv: "040efa547882587f0935e8631ee85531"
server_key_version: "encryptionkey@0"
I20260811 22:14:21.607607 23311 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/wal/instance:
uuid: "1fcb604df58a41fb8759726b378900c2"
format_stamp: "Formatted at 2026-08-11 22:14:21 on dist-test-slave-vxj2"
server_key: "a2b712cad15c4d48526225a113da5577"
server_key_iv: "040efa547882587f0935e8631ee85531"
server_key_version: "encryptionkey@0"
I20260811 22:14:21.612032 23311 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.003s
I20260811 22:14:21.615060 23329 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:21.616617 23311 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260811 22:14:21.616817 23311 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/wal
uuid: "1fcb604df58a41fb8759726b378900c2"
format_stamp: "Formatted at 2026-08-11 22:14:21 on dist-test-slave-vxj2"
server_key: "a2b712cad15c4d48526225a113da5577"
server_key_iv: "040efa547882587f0935e8631ee85531"
server_key_version: "encryptionkey@0"
I20260811 22:14:21.616971 23311 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:21.623713 23214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } attempt: 1
I20260811 22:14:21.624593 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.624265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:33873 (local address 127.0.0.1:51070)
0811 22:14:21.624436 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.624504 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
I20260811 22:14:21.625450 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.625205 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:33873 (local address 127.0.0.1:51084)
0811 22:14:21.625327 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.625386 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260811 22:14:21.625725 23214 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:33873: Network error: Client connection negotiation failed: client connection to 127.28.47.188:33873: connect: Connection refused (error 111)
I20260811 22:14:21.651261 23311 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:21.654735 23311 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:21.654990 23311 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:21.663318 23311 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.188:33873
I20260811 22:14:21.663610 23384 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.188:33873 every 8 connection(s)
I20260811 22:14:21.664379 23311 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/data/info.pb
I20260811 22:14:21.666465 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 23311
I20260811 22:14:21.666575 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786486147647525-28862-0/minicluster-data/master-2/wal/instance
I20260811 22:14:21.666896 28862 external_mini_cluster.cc:1428] Setting key 889d38e0fb76676278480f8b39f07f5d
I20260811 22:14:21.667614 23385 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:21.669629 23305 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } attempt: 3
I20260811 22:14:21.672936 23385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486461, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.189@KRBTEST.COM for kudu/127.28.47.188@KRBTEST.COM
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.188@KRBTEST.COM
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486461, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.188@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:14:21.683068 23308 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.670095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:33873 (local address 127.0.0.1:51116)
0811 22:14:21.670302 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.670321 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:21.670405 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:21.671084 (+ 679us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:21.671088 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:21.671310 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:21.671418 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.671425 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.673832 (+ 2407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.673836 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.674608 (+ 772us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.674623 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.675507 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.675511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.675600 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.676071 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:21.676088 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:21.677561 (+ 1473us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:21.681267 (+ 3706us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.681271 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.681273 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.681539 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.681867 (+ 328us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.681874 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.681879 (+ 5us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.681987 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.682468 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:21.682471 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:21.682608 (+ 137us) client_negotiation.cc:769] Sending connection context
0811 22:14:21.682735 (+ 127us) client_negotiation.cc:240] Negotiation successful
0811 22:14:21.682897 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125}
I20260811 22:14:21.684319 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.670149 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51116 (local address 127.28.47.188:33873)
0811 22:14:21.670865 (+ 716us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.670871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.670882 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:14:21.670910 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.670911 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.670941 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.671046 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.672859 (+ 1813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.673681 (+ 822us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.674764 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.675333 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.675839 (+ 506us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.677747 (+ 1908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.677754 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.677766 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.677797 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.681122 (+ 3325us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.681639 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.681644 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.681645 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.681755 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.682087 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.682092 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.682094 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.682354 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.682491 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.682918 (+ 427us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.683542 (+ 624us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":658,"spinlock_wait_cycles":9600,"thread_start_us":106,"threads_started":1}
I20260811 22:14:21.684329 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.668315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51100 (local address 127.28.47.188:33873)
0811 22:14:21.668767 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.668785 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.668823 (+ 38us) server_negotiation.cc:407] Connection header received
0811 22:14:21.669577 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.669600 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.670019 (+ 419us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.670408 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.671599 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.672700 (+ 1101us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.674065 (+ 1365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.674578 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.674760 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.678378 (+ 3618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.678382 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.678384 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.678410 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.680689 (+ 2279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.681184 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.681191 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.681197 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.681368 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.681734 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.681737 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.681739 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.682361 (+ 622us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.682551 (+ 190us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.682939 (+ 388us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.683227 (+ 288us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":123,"server-negotiator.queue_time_us":299,"thread_start_us":158,"threads_started":1}
I20260811 22:14:21.687080 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.675333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35736 (local address 127.28.47.190:45825)
0811 22:14:21.675569 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.675576 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.676335 (+ 759us) server_negotiation.cc:407] Connection header received
0811 22:14:21.676484 (+ 149us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.676490 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.676554 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.676651 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.677437 (+ 786us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.678140 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.679411 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.679798 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.679928 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.683110 (+ 3182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.683114 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.683117 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.683145 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.684893 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.685462 (+ 569us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.685465 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.685467 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.685560 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.685986 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.685990 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.685991 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.686190 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.686331 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.686791 (+ 460us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.686920 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125}
I20260811 22:14:21.687386 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.675275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:45825 (local address 127.0.0.1:35736)
0811 22:14:21.676149 (+ 874us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.676216 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:21.676379 (+ 163us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:21.676747 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:21.676751 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:21.677094 (+ 343us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:21.677314 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.677322 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.678251 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.678256 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.679257 (+ 1001us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.679280 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.679941 (+ 661us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.679945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.680066 (+ 121us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.680532 (+ 466us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:21.680550 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:21.682946 (+ 2396us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:21.685010 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.685018 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.685022 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.685346 (+ 324us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.685694 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.685698 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.685699 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.685876 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.686330 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:21.686337 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:21.686477 (+ 140us) client_negotiation.cc:769] Sending connection context
0811 22:14:21.686699 (+ 222us) client_negotiation.cc:240] Negotiation successful
0811 22:14:21.687175 (+ 476us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":386,"thread_start_us":171,"threads_started":1}
I20260811 22:14:21.688647 23385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:14:21.689613 23305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6: Bootstrap starting.
I20260811 22:14:21.692759 23305 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:21.692894 23214 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } attempt: 2
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486461, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.188@KRBTEST.COM for kudu/127.28.47.189@KRBTEST.COM
I20260811 22:14:21.694043 23305 log.cc:824] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6: Log is configured to *not* fsync() on all Append() calls
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:14:21.696692 23305 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6: No bootstrap required, opened a new log
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486461, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for kudu/127.28.47.188@KRBTEST.COM
I20260811 22:14:21.699003 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.687587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35748 (local address 127.28.47.190:45825)
0811 22:14:21.687946 (+ 359us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.687950 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.687962 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:14:21.688000 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.688002 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.688036 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.688097 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.688908 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.689647 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.690728 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.691061 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.691143 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.694994 (+ 3851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.694999 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.695003 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.695026 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.696674 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.697312 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.697319 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.697321 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.697421 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.697764 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.697767 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.697769 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.698038 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.698307 (+ 269us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.698700 (+ 393us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.698838 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":283}
I20260811 22:14:21.699750 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.689302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33016 (local address 127.28.47.189:37013)
0811 22:14:21.689422 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.689428 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.689444 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:14:21.689554 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.689556 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.689596 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.689691 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.690157 (+ 466us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.690645 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.691403 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.691697 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.691805 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.694835 (+ 3030us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.694840 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.694843 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.694872 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.697625 (+ 2753us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.698235 (+ 610us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.698238 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.698240 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.698337 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.698683 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.698686 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.698689 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.698970 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.699050 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.699474 (+ 424us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.699604 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260811 22:14:21.700614 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.689211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:37013 (local address 127.0.0.1:33016)
0811 22:14:21.689384 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.689398 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:21.689475 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:21.689785 (+ 310us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:21.689788 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:21.689942 (+ 154us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:21.690037 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.690041 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.690755 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.690768 (+ 13us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.691285 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.691293 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.691952 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.691957 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.692042 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.692480 (+ 438us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:21.692501 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:21.694669 (+ 2168us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:21.697843 (+ 3174us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.697847 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.697849 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.698117 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.698467 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.698470 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.698472 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.698569 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.699139 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:21.699141 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:21.699252 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 22:14:21.700231 (+ 979us) client_negotiation.cc:240] Negotiation successful
0811 22:14:21.700414 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105}
I20260811 22:14:21.702183 23385 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:14:21.702334 23305 raft_consensus.cc:348] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } }
I20260811 22:14:21.702611 23305 raft_consensus.cc:729] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6c0f06b73604170bbd8cd3d75c5aca6, State: Initialized, Role: FOLLOWER
I20260811 22:14:21.703008 23305 consensus_queue.cc:260] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } }
I20260811 22:14:21.704377 23217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.693361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:33873 (local address 127.0.0.1:51132)
0811 22:14:21.693584 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.693600 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:21.693680 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:21.694182 (+ 502us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:21.694185 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:21.694381 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:21.694487 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.694495 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.695571 (+ 1076us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.695576 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.696535 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.696552 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.697194 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.697198 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.697280 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.697752 (+ 472us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:21.697777 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:21.699728 (+ 1951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:21.702004 (+ 2276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.702010 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.702014 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.702287 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.703293 (+ 1006us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.703297 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.703300 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.703404 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.703866 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:21.703869 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:21.703968 (+ 99us) client_negotiation.cc:769] Sending connection context
0811 22:14:21.704072 (+ 104us) client_negotiation.cc:240] Negotiation successful
0811 22:14:21.704213 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260811 22:14:21.704700 23305 sys_catalog.cc:564] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:21.705034 23388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.693423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51132 (local address 127.28.47.188:33873)
0811 22:14:21.693854 (+ 431us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.693861 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.693876 (+ 15us) server_negotiation.cc:407] Connection header received
0811 22:14:21.693912 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.693915 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.693956 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.694050 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.694628 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.695425 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.696696 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.697035 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.697164 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.700085 (+ 2921us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.700089 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.700092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.700120 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.701883 (+ 1763us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.702409 (+ 526us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.702412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.702414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.703054 (+ 640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.703521 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.703524 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.703526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.703727 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.703818 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.704702 (+ 884us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.704897 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337}
I20260811 22:14:21.705893 23392 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } } }
I20260811 22:14:21.706136 23392 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6 [sys.catalog]: This master's current role is: FOLLOWER
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486461, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.188@KRBTEST.COM for kudu/127.28.47.188@KRBTEST.COM
Aug 11 22:14:21 dist-test-slave-vxj2 krb5kdc[23126](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486460, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.189@KRBTEST.COM
I20260811 22:14:21.709009 23214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477: Bootstrap starting.
I20260811 22:14:21.712180 23214 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:21.712236 23390 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.702705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:33873 (local address 127.0.0.1:51140)
0811 22:14:21.702867 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:21.702884 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:21.703049 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:21.703369 (+ 320us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:21.703372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:21.703538 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:21.703621 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.703627 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.704527 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.704530 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.705322 (+ 792us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:21.705331 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.705987 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.705990 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:21.706066 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.706603 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:21.706627 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:21.708338 (+ 1711us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:21.710643 (+ 2305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.710646 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.710648 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.710888 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.711175 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:21.711178 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:21.711179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:21.711276 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:21.711653 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:21.711655 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:21.711757 (+ 102us) client_negotiation.cc:769] Sending connection context
0811 22:14:21.711873 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 22:14:21.712049 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 22:14:21.712538 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.702904 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51140 (local address 127.28.47.188:33873)
0811 22:14:21.702980 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.702983 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.702995 (+ 12us) server_negotiation.cc:407] Connection header received
0811 22:14:21.703160 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.703164 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.703211 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.703258 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.703725 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.704417 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.705430 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.705773 (+ 343us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.705847 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.708478 (+ 2631us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.708482 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.708485 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.708509 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.710512 (+ 2003us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.710997 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.711001 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.711003 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.711080 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.711350 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.711354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.711355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.711550 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.711664 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.711935 (+ 271us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.712114 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":70,"server-negotiator.queue_time_us":27}
I20260811 22:14:21.713285 23214 log.cc:824] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:21.713620 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.701609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33030 (local address 127.28.47.189:37013)
0811 22:14:21.701848 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.701852 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.701863 (+ 11us) server_negotiation.cc:407] Connection header received
0811 22:14:21.701903 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.701905 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.701963 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.702036 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.703203 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.703994 (+ 791us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.705069 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.705434 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.705502 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.709315 (+ 3813us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.709322 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.709326 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.709361 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.711167 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.711708 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.711711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.711713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.711822 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.712187 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.712193 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.712197 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.712455 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.712662 (+ 207us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.713199 (+ 537us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.713468 (+ 269us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260811 22:14:21.715899 23214 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477: No bootstrap required, opened a new log
I20260811 22:14:21.716590 23385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2: Bootstrap starting.
I20260811 22:14:21.717417 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:21.718892 23185 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "193a033c826d43faaaec04daedd80477"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35748
I20260811 22:14:21.720052 23385 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:21.720304 23214 raft_consensus.cc:348] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } }
I20260811 22:14:21.720567 23214 raft_consensus.cc:729] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 193a033c826d43faaaec04daedd80477, State: Initialized, Role: FOLLOWER
I20260811 22:14:21.720932 23214 consensus_queue.cc:260] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } }
I20260811 22:14:21.721403 23385 log.cc:824] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:21.717361 23221 master_runner.cc:186] Error getting master registration for 127.28.47.190:45825: OK, instance_id { permanent_uuid: "193a033c826d43faaaec04daedd80477" instance_seqno: 1786486461156816 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:14:21.721630 23221 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:14:21.721900 23407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } } }
I20260811 22:14:21.722045 23407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:14:21.722395 23214 sys_catalog.cc:564] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:21.722456 23185 raft_consensus.cc:482] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:14:21.722606 23185 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:21.723879 23185 raft_consensus.cc:504] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } }
I20260811 22:14:21.724195 23385 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2: No bootstrap required, opened a new log
I20260811 22:14:21.724864 23185 leader_election.cc:288] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [CANDIDATE]: Term 1 election: Requested vote from peers f6c0f06b73604170bbd8cd3d75c5aca6 (127.28.47.189:37013), 1fcb604df58a41fb8759726b378900c2 (127.28.47.188:33873)
W20260811 22:14:21.727648 23421 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:14:21.727741 23421 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:14:21.728482 23270 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "193a033c826d43faaaec04daedd80477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6"
I20260811 22:14:21.728736 23270 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:21.728662 23385 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } }
I20260811 22:14:21.728909 23385 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1fcb604df58a41fb8759726b378900c2, State: Initialized, Role: FOLLOWER
I20260811 22:14:21.729281 23385 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } }
I20260811 22:14:21.730243 23270 raft_consensus.cc:2454] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a033c826d43faaaec04daedd80477 in term 1.
I20260811 22:14:21.730706 23385 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:21.730742 23357 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "193a033c826d43faaaec04daedd80477" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "1fcb604df58a41fb8759726b378900c2"
I20260811 22:14:21.730955 23357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:21.730986 23158 leader_election.cc:302] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 193a033c826d43faaaec04daedd80477, f6c0f06b73604170bbd8cd3d75c5aca6; no voters:
I20260811 22:14:21.733016 23357 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 193a033c826d43faaaec04daedd80477 in term 1.
I20260811 22:14:21.732729 23140 master_runner.cc:186] Error getting master registration for 127.28.47.188:33873: OK, instance_id { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" instance_seqno: 1786486461661234 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:14:21.736105 23140 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:14:21.737408 23407 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:21.737752 23407 raft_consensus.cc:686] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [term 1 LEADER]: Becoming Leader. State: Replica: 193a033c826d43faaaec04daedd80477, State: Running, Role: LEADER
I20260811 22:14:21.738140 23407 consensus_queue.cc:237] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } }
W20260811 22:14:21.738266 23406 catalog_manager.cc:1558] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:14:21.738341 23406 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:14:21.739400 23311 master_runner.cc:263] No leader master found from master 1fcb604df58a41fb8759726b378900c2
I20260811 22:14:21.739485 23311 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:14:21.740036 23407 sys_catalog.cc:455] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 193a033c826d43faaaec04daedd80477. Latest consensus state: current_term: 1 leader_uuid: "193a033c826d43faaaec04daedd80477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } } }
I20260811 22:14:21.740164 23407 sys_catalog.cc:458] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:21.740535 23448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:21.741604 23448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:21.741863 23429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } } }
I20260811 22:14:21.742013 23429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2 [sys.catalog]: This master's current role is: FOLLOWER
W20260811 22:14:21.748880 23444 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:14:21.748986 23444 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:14:21.751485 23389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.728173 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51152 (local address 127.28.47.188:33873)
0811 22:14:21.728336 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.728343 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.741481 (+ 13138us) server_negotiation.cc:407] Connection header received
0811 22:14:21.741704 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.741708 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.741771 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.741849 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.742976 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.743770 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.744981 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.745391 (+ 410us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.745478 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.747616 (+ 2138us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.747620 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.747623 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.747651 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.749504 (+ 1853us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.750049 (+ 545us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.750053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.750055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.750146 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.750446 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.750449 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.750451 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.750680 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.750812 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.751188 (+ 376us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.751318 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260811 22:14:21.767390 23218 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.745579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35764 (local address 127.28.47.190:45825)
0811 22:14:21.745701 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.745706 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.753378 (+ 7672us) server_negotiation.cc:407] Connection header received
0811 22:14:21.753555 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.753558 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.753611 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.753670 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.754573 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.755390 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.756548 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.756899 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.756973 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.762395 (+ 5422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.762399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.762402 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.762436 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.764249 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.764785 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.764789 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.764791 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.764881 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.765216 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.765220 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.765222 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.765460 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.765562 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.767073 (+ 1511us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.767212 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260811 22:14:21.776014 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:21.746311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33044 (local address 127.28.47.189:37013)
0811 22:14:21.746444 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.746469 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.757900 (+ 11431us) server_negotiation.cc:407] Connection header received
0811 22:14:21.759415 (+ 1515us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.759419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.759475 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.759528 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.760363 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.761177 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.765904 (+ 4727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:21.766259 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:21.766339 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:21.772079 (+ 5740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:21.772084 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:21.772087 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:21.772122 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:21.773960 (+ 1838us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.774525 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.774529 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.774531 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.774631 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:21.774949 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:21.774953 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:21.774955 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:21.775194 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:21.775323 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:21.775692 (+ 369us) server_negotiation.cc:299] Negotiation successful
0811 22:14:21.775841 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 22:14:21.780435 23357 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2 [term 1 FOLLOWER]: Refusing update from remote peer 193a033c826d43faaaec04daedd80477: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:21.780485 23269 raft_consensus.cc:1261] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6 [term 1 FOLLOWER]: Refusing update from remote peer 193a033c826d43faaaec04daedd80477: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:21.781949 23407 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:21.782231 23453 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:21.788930 23429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 193a033c826d43faaaec04daedd80477. Latest consensus state: current_term: 1 leader_uuid: "193a033c826d43faaaec04daedd80477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } } }
I20260811 22:14:21.789099 23429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:14:21.793646 23392 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 193a033c826d43faaaec04daedd80477. Latest consensus state: current_term: 1 leader_uuid: "193a033c826d43faaaec04daedd80477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } } }
I20260811 22:14:21.793798 23392 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:14:21.804416 23453 sys_catalog.cc:455] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "193a033c826d43faaaec04daedd80477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } } }
I20260811 22:14:21.804598 23453 sys_catalog.cc:458] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:21.805619 23446 sys_catalog.cc:455] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "193a033c826d43faaaec04daedd80477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } } }
I20260811 22:14:21.805778 23446 sys_catalog.cc:458] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:21.806083 23392 sys_catalog.cc:455] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "193a033c826d43faaaec04daedd80477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } } }
I20260811 22:14:21.806208 23392 sys_catalog.cc:458] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:14:21.807250 23429 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "193a033c826d43faaaec04daedd80477" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "193a033c826d43faaaec04daedd80477" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 45825 } } peers { permanent_uuid: "f6c0f06b73604170bbd8cd3d75c5aca6" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 37013 } } peers { permanent_uuid: "1fcb604df58a41fb8759726b378900c2" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 33873 } } }
I20260811 22:14:21.807384 23429 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:14:21.810331 23456 mvcc.cc:205] Tried to move back new op lower bound from 7317448547443318784 to 7317448547283374080. Current Snapshot: MvccSnapshot[applied={T|T < 7317448547443318784}]
I20260811 22:14:21.810590 23454 mvcc.cc:205] Tried to move back new op lower bound from 7317448547443318784 to 7317448547283374080. Current Snapshot: MvccSnapshot[applied={T|T < 7317448547443318784}]
I20260811 22:14:21.817723 23448 catalog_manager.cc:1347] Generated new cluster ID: 17b788c0fe404e6e91cf905847078360
I20260811 22:14:21.817816 23448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:21.836355 23448 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:21.838016 23448 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:21.852758 23448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 193a033c826d43faaaec04daedd80477: Generated new TSK 0
I20260811 22:14:21.853768 23448 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:14:21.888696 23389 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:21.885609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51168 (local address 127.28.47.188:33873)
0811 22:14:21.885780 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.885785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.887235 (+ 1450us) server_negotiation.cc:407] Connection header received
0811 22:14:21.887707 (+ 472us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.887711 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.887771 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.887825 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.888545 (+ 720us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:51168: BlockingRecv error: recv got EOF from 127.0.0.1:51168 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 22:14:21.890272 23218 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:21.886161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35768 (local address 127.28.47.190:45825)
0811 22:14:21.886283 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.886288 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.889249 (+ 2961us) server_negotiation.cc:407] Connection header received
0811 22:14:21.889432 (+ 183us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.889435 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.889486 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.889541 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.890142 (+ 601us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35768: BlockingRecv error: recv got EOF from 127.0.0.1:35768 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 22:14:21.891707 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:21.886865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33048 (local address 127.28.47.189:37013)
0811 22:14:21.886987 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.886991 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.890667 (+ 3676us) server_negotiation.cc:407] Connection header received
0811 22:14:21.890961 (+ 294us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.890967 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.891019 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.891071 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.891593 (+ 522us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33048: BlockingRecv error: recv got EOF from 127.0.0.1:33048 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260811 22:14:21.896884 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:21.895222 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33064 (local address 127.28.47.189:37013)
0811 22:14:21.895343 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.895347 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.895817 (+ 470us) server_negotiation.cc:407] Connection header received
0811 22:14:21.896067 (+ 250us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.896071 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.896152 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.896208 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.896759 (+ 551us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33064: BlockingRecv error: recv got EOF from 127.0.0.1:33064 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 22:14:21.902457 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:21.900734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33068 (local address 127.28.47.189:37013)
0811 22:14:21.900864 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.900868 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.901420 (+ 552us) server_negotiation.cc:407] Connection header received
0811 22:14:21.901641 (+ 221us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.901645 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.901707 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.901766 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.902347 (+ 581us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33068: BlockingRecv error: recv got EOF from 127.0.0.1:33068 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
W20260811 22:14:21.908939 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:21.907123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33076 (local address 127.28.47.189:37013)
0811 22:14:21.907251 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.907256 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.907860 (+ 604us) server_negotiation.cc:407] Connection header received
0811 22:14:21.908083 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.908088 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.908150 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.908203 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.908819 (+ 616us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33076: BlockingRecv error: recv got EOF from 127.0.0.1:33076 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260811 22:14:21.919351 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:21.917348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33084 (local address 127.28.47.189:37013)
0811 22:14:21.917531 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.917537 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.918167 (+ 630us) server_negotiation.cc:407] Connection header received
0811 22:14:21.918436 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.918440 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.918506 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.918565 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.919210 (+ 645us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33084: BlockingRecv error: recv got EOF from 127.0.0.1:33084 (error 108)
Metrics: {"server-negotiator.queue_time_us":97}
W20260811 22:14:21.932739 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:21.930339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33088 (local address 127.28.47.189:37013)
0811 22:14:21.930484 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.930490 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.931193 (+ 703us) server_negotiation.cc:407] Connection header received
0811 22:14:21.931483 (+ 290us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.931487 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.931594 (+ 107us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.931666 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.932609 (+ 943us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33088: BlockingRecv error: recv got EOF from 127.0.0.1:33088 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
W20260811 22:14:21.953447 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:21.951681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33096 (local address 127.28.47.189:37013)
0811 22:14:21.951816 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.951821 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.952368 (+ 547us) server_negotiation.cc:407] Connection header received
0811 22:14:21.952583 (+ 215us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.952587 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.952642 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.952694 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.953316 (+ 622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33096: BlockingRecv error: recv got EOF from 127.0.0.1:33096 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260811 22:14:21.989928 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:21.988523 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33112 (local address 127.28.47.189:37013)
0811 22:14:21.988737 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:21.988742 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:21.988910 (+ 168us) server_negotiation.cc:407] Connection header received
0811 22:14:21.989119 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:21.989123 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:21.989211 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:21.989321 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:21.989824 (+ 503us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33112: BlockingRecv error: recv got EOF from 127.0.0.1:33112 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260811 22:14:22.058427 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:22.056962 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33126 (local address 127.28.47.189:37013)
0811 22:14:22.057196 (+ 234us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:22.057201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:22.057338 (+ 137us) server_negotiation.cc:407] Connection header received
0811 22:14:22.057543 (+ 205us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:22.057551 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:22.057606 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:22.057707 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:22.058268 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33126: BlockingRecv error: recv got EOF from 127.0.0.1:33126 (error 108)
Metrics: {"server-negotiator.queue_time_us":93}
W20260811 22:14:22.190387 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:22.189015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33130 (local address 127.28.47.189:37013)
0811 22:14:22.189212 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:22.189219 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:22.189345 (+ 126us) server_negotiation.cc:407] Connection header received
0811 22:14:22.189553 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:22.189556 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:22.189609 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:22.189706 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:22.190235 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33130: BlockingRecv error: recv got EOF from 127.0.0.1:33130 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
W20260811 22:14:22.450284 23310 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:22.448958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33146 (local address 127.28.47.189:37013)
0811 22:14:22.449114 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:22.449121 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:22.449203 (+ 82us) server_negotiation.cc:407] Connection header received
0811 22:14:22.449452 (+ 249us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:22.449455 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:22.449506 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:22.449601 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:22.450135 (+ 534us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33146: BlockingRecv error: recv got EOF from 127.0.0.1:33146 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
I20260811 22:14:22.739890 23406 catalog_manager.cc:1259] Loaded cluster ID: 17b788c0fe404e6e91cf905847078360
I20260811 22:14:22.740015 23406 catalog_manager.cc:1552] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6: loading cluster ID for follower catalog manager: success
I20260811 22:14:22.741842 23406 catalog_manager.cc:1574] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6: acquiring CA information for follower catalog manager: success
I20260811 22:14:22.743184 23406 catalog_manager.cc:1602] T 00000000000000000000000000000000 P f6c0f06b73604170bbd8cd3d75c5aca6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:14:22.750557 23444 catalog_manager.cc:1259] Loaded cluster ID: 17b788c0fe404e6e91cf905847078360
I20260811 22:14:22.750653 23444 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2: loading cluster ID for follower catalog manager: success
I20260811 22:14:22.752609 23444 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2: acquiring CA information for follower catalog manager: success
I20260811 22:14:22.754045 23444 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 1fcb604df58a41fb8759726b378900c2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:14:22.969331 23534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:22.965041 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33158 (local address 127.28.47.189:37013)
0811 22:14:22.965342 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:22.965348 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:22.965465 (+ 117us) server_negotiation.cc:407] Connection header received
0811 22:14:22.965656 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:22.965658 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:22.965796 (+ 138us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:22.965920 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:22.966210 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:22.966779 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:22.967680 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:22.967928 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:22.968039 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:22.968400 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:22.968872 (+ 472us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:22.969021 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:22.969058 (+ 37us) server_negotiation.cc:299] Negotiation successful
0811 22:14:22.969126 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":105,"threads_started":1}
I20260811 22:14:22.977474 23540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:22.973066 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51172 (local address 127.28.47.188:33873)
0811 22:14:22.973365 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:22.973369 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:22.973386 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:14:22.973529 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:22.973531 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:22.973659 (+ 128us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:22.973772 (+ 113us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:22.974136 (+ 364us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:22.974809 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:22.975768 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:22.976017 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:22.976147 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:22.976484 (+ 337us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:22.976962 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:22.977063 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:22.977269 (+ 206us) server_negotiation.cc:299] Negotiation successful
0811 22:14:22.977341 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":125,"threads_started":1}
I20260811 22:14:22.985879 23534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:22.981619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33160 (local address 127.28.47.189:37013)
0811 22:14:22.981756 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:22.981761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:22.981958 (+ 197us) server_negotiation.cc:407] Connection header received
0811 22:14:22.982054 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:22.982060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:22.982140 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:22.982271 (+ 131us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:22.982541 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:22.983339 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:22.984491 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:22.984849 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:22.984953 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:22.985319 (+ 366us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:22.985404 (+ 85us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:22.985490 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:22.985646 (+ 156us) server_negotiation.cc:299] Negotiation successful
0811 22:14:22.985723 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:14:22.985880 23540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:22.981011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51176 (local address 127.28.47.188:33873)
0811 22:14:22.981140 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:22.981147 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:22.981211 (+ 64us) server_negotiation.cc:407] Connection header received
0811 22:14:22.981387 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:22.981390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:22.981481 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:22.981555 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:22.982082 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:22.983030 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:22.984372 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:22.984661 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:22.984780 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:22.985101 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:22.985240 (+ 139us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:22.985310 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:22.985638 (+ 328us) server_negotiation.cc:299] Negotiation successful
0811 22:14:22.985699 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260811 22:14:22.986461 23546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:22.980892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35770 (local address 127.28.47.190:45825)
0811 22:14:22.981225 (+ 333us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:22.981230 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:22.981616 (+ 386us) server_negotiation.cc:407] Connection header received
0811 22:14:22.981734 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:22.981737 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:22.981934 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:22.982088 (+ 154us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:22.982352 (+ 264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:22.982956 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:22.984375 (+ 1419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:22.984739 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:22.984839 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:22.985375 (+ 536us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:22.985760 (+ 385us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:22.985854 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:22.985979 (+ 125us) server_negotiation.cc:299] Negotiation successful
0811 22:14:22.986293 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":210,"thread_start_us":76,"threads_started":1}
I20260811 22:14:22.990054 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 23140
I20260811 22:14:23.001950 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 23221
I20260811 22:14:23.011759 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 23311
2026-08-11T22:14:23Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3330 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 11 22:14:23 dist-test-slave-vxj2 krb5kdc[23553](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 11 22:14:23 dist-test-slave-vxj2 krb5kdc[23553](info): set up 2 sockets
Aug 11 22:14:23 dist-test-slave-vxj2 krb5kdc[23553](info): commencing operation
krb5kdc: starting...
W20260811 22:14:24.181035 28862 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/test-user.keytab.
Aug 11 22:14:24 dist-test-slave-vxj2 krb5kdc[23553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-11T22:14:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-11T22:14:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.28.47.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
Entry for principal kudu/127.28.47.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab.
I20260811 22:14:24.329052 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:34757
--webserver_interface=127.28.47.190
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:34757,127.28.47.189:42943,127.28.47.188:46239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260811 22:14:24.450503 23567 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:14:24.450969 23567 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:24.451050 23567 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:24.451134 23567 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:24.455641 23567 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:24.455763 23567 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:24.455801 23567 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:24.455840 23567 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:24.455878 23567 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:24.461845 23567 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:34757,127.28.47.189:42943,127.28.47.188:46239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.190.keytab
--principal=kudu/127.28.47.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.190:34757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.28.47.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23567
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:24.463143 23567 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:24.464289 23567 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:24.473815 23574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:24.473870 23576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:24.473775 23580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:24.474252 23567 server_base.cc:1034] running on GCE node
I20260811 22:14:24.474951 23567 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:24.476270 23567 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:24.477474 23567 hybrid_clock.cc:648] HybridClock initialized: now 1786486464477461 us; error 61 us; skew 500 ppm
Aug 11 22:14:24 dist-test-slave-vxj2 krb5kdc[23553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:24.480561 23567 init.cc:377] Logged in from keytab as kudu/127.28.47.190@KRBTEST.COM (short username kudu)
I20260811 22:14:24.481734 23567 webserver.cc:460] Webserver started at http://127.28.47.190:36041/ using document root <none> and password file <none>
I20260811 22:14:24.482402 23567 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:24.482508 23567 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:24.482764 23567 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:24.485083 23567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/data/instance:
uuid: "1a44720fd92443ad9a407675799fd9a9"
format_stamp: "Formatted at 2026-08-11 22:14:24 on dist-test-slave-vxj2"
server_key: "bce824387098f977be7abc9c74e322ab"
server_key_iv: "99769c37b9eaf79a5cf045e3779d51e6"
server_key_version: "encryptionkey@0"
I20260811 22:14:24.485810 23567 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/wal/instance:
uuid: "1a44720fd92443ad9a407675799fd9a9"
format_stamp: "Formatted at 2026-08-11 22:14:24 on dist-test-slave-vxj2"
server_key: "bce824387098f977be7abc9c74e322ab"
server_key_iv: "99769c37b9eaf79a5cf045e3779d51e6"
server_key_version: "encryptionkey@0"
I20260811 22:14:24.490131 23567 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260811 22:14:24.492882 23587 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:24.494426 23567 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:24.494598 23567 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/wal
uuid: "1a44720fd92443ad9a407675799fd9a9"
format_stamp: "Formatted at 2026-08-11 22:14:24 on dist-test-slave-vxj2"
server_key: "bce824387098f977be7abc9c74e322ab"
server_key_iv: "99769c37b9eaf79a5cf045e3779d51e6"
server_key_version: "encryptionkey@0"
I20260811 22:14:24.494735 23567 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:24.512128 23567 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:24.515595 23567 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:24.515842 23567 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:24.524055 23567 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.190:34757
I20260811 22:14:24.524410 23641 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.190:34757 every 8 connection(s)
I20260811 22:14:24.525435 23567 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/data/info.pb
I20260811 22:14:24.527730 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 23567
I20260811 22:14:24.527908 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-0/wal/instance
I20260811 22:14:24.528232 28862 external_mini_cluster.cc:1428] Setting key 96c20e125ab2d35d945096b65ec90881
I20260811 22:14:24.528865 23642 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:24.533280 23642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.28.47.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.189@KRBTEST.COM" created.
Aug 11 22:14:24 dist-test-slave-vxj2 krb5kdc[23553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:14:24.551004 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.535950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:34757 (local address 127.0.0.1:60378)
0811 22:14:24.536628 (+ 678us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.536666 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:24.537647 (+ 981us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:24.538461 (+ 814us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:24.538466 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:24.538826 (+ 360us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:24.539097 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:24.539111 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.540493 (+ 1382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.540500 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:24.541486 (+ 986us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:24.541499 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.542548 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.542554 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:24.542715 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:24.543381 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:24.543408 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:24.545332 (+ 1924us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:24.547540 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:24.547550 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:24.547561 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:24.547813 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:24.548162 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:24.548165 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:24.548167 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:24.548293 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:24.548953 (+ 660us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:24.548960 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:24.549138 (+ 178us) client_negotiation.cc:769] Sending connection context
0811 22:14:24.549383 (+ 245us) client_negotiation.cc:240] Negotiation successful
0811 22:14:24.549882 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"mutex_wait_us":175,"spinlock_wait_cycles":4864,"thread_start_us":175,"threads_started":1}
I20260811 22:14:24.551004 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.535949 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60378 (local address 127.28.47.190:34757)
0811 22:14:24.536480 (+ 531us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:24.536501 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:24.536793 (+ 292us) server_negotiation.cc:407] Connection header received
0811 22:14:24.537983 (+ 1190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:24.537988 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:24.538187 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:24.538408 (+ 221us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:24.539283 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.540350 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.541628 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.542313 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.542533 (+ 220us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:24.545509 (+ 2976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:24.545515 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:24.545528 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:24.545576 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:24.547402 (+ 1826us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:24.547942 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:24.547947 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:24.547949 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:24.548056 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:24.548392 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:24.548395 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:24.548397 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:24.548832 (+ 435us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:24.548985 (+ 153us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:24.549464 (+ 479us) server_negotiation.cc:299] Negotiation successful
0811 22:14:24.549830 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":21,"server-negotiator.queue_time_us":363,"thread_start_us":179,"threads_started":1}
I20260811 22:14:24.554046 23642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:14:24.554949 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.554615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:42943 (local address 127.0.0.1:56972)
0811 22:14:24.554765 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.554861 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:42943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260811 22:14:24.555244 23588 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.47.189:42943: connect: Connection refused (error 111)
I20260811 22:14:24.556294 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.556036 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:42943 (local address 127.0.0.1:56984)
0811 22:14:24.556163 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.556223 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:42943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
W20260811 22:14:24.556461 23642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.189:42943: Network error: Client connection negotiation failed: client connection to 127.28.47.189:42943: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.189.keytab.
Entry for principal kudu/127.28.47.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.189.keytab.
I20260811 22:14:24.561432 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.189:42943
--webserver_interface=127.28.47.189
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.189.keytab
--principal=kudu/127.28.47.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:34757,127.28.47.189:42943,127.28.47.188:46239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:14:24.605803 23642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } attempt: 1
I20260811 22:14:24.606791 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.606457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:42943 (local address 127.0.0.1:56996)
0811 22:14:24.606617 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.606690 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:42943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260811 22:14:24.607748 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.607490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:42943 (local address 127.0.0.1:57006)
0811 22:14:24.607624 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.607684 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:42943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260811 22:14:24.607960 23642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.189:42943: Network error: Client connection negotiation failed: client connection to 127.28.47.189:42943: connect: Connection refused (error 111)
I20260811 22:14:24.676208 23642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } attempt: 2
I20260811 22:14:24.677338 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.676958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:42943 (local address 127.0.0.1:57008)
0811 22:14:24.677130 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.677250 (+ 120us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:42943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260811 22:14:24.678233 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.677982 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:42943 (local address 127.0.0.1:57014)
0811 22:14:24.678114 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.678172 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.189:42943: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260811 22:14:24.678387 23642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.189:42943: Network error: Client connection negotiation failed: client connection to 127.28.47.189:42943: connect: Connection refused (error 111)
W20260811 22:14:24.681861 23649 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:14:24.682232 23649 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:24.682301 23649 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:24.682408 23649 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:24.686691 23649 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:24.686785 23649 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:24.686828 23649 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:24.686887 23649 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:24.686926 23649 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:24.692585 23649 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:34757,127.28.47.189:42943,127.28.47.188:46239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.189.keytab
--principal=kudu/127.28.47.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.189:42943
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.28.47.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23649
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:24.693889 23649 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:24.694972 23649 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:24.704015 23655 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:24.704110 23656 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:24.704177 23661 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:24.704061 23649 server_base.cc:1034] running on GCE node
I20260811 22:14:24.704764 23649 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:24.706046 23649 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:24.707243 23649 hybrid_clock.cc:648] HybridClock initialized: now 1786486464707209 us; error 67 us; skew 500 ppm
Aug 11 22:14:24 dist-test-slave-vxj2 krb5kdc[23553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:24.714247 23649 init.cc:377] Logged in from keytab as kudu/127.28.47.189@KRBTEST.COM (short username kudu)
I20260811 22:14:24.715369 23649 webserver.cc:460] Webserver started at http://127.28.47.189:39829/ using document root <none> and password file <none>
I20260811 22:14:24.715992 23649 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:24.716091 23649 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:24.716343 23649 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:24.718662 23649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/data/instance:
uuid: "7fa8399bdfb340dd98517a0baf26bb09"
format_stamp: "Formatted at 2026-08-11 22:14:24 on dist-test-slave-vxj2"
server_key: "299ac318cd168dccaefcf790a20d97f3"
server_key_iv: "ef3d92181bd6b3923784e3afb16a6f36"
server_key_version: "encryptionkey@0"
I20260811 22:14:24.719316 23649 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/wal/instance:
uuid: "7fa8399bdfb340dd98517a0baf26bb09"
format_stamp: "Formatted at 2026-08-11 22:14:24 on dist-test-slave-vxj2"
server_key: "299ac318cd168dccaefcf790a20d97f3"
server_key_iv: "ef3d92181bd6b3923784e3afb16a6f36"
server_key_version: "encryptionkey@0"
I20260811 22:14:24.723393 23649 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260811 22:14:24.726094 23669 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:24.727464 23649 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:24.727624 23649 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/wal
uuid: "7fa8399bdfb340dd98517a0baf26bb09"
format_stamp: "Formatted at 2026-08-11 22:14:24 on dist-test-slave-vxj2"
server_key: "299ac318cd168dccaefcf790a20d97f3"
server_key_iv: "ef3d92181bd6b3923784e3afb16a6f36"
server_key_version: "encryptionkey@0"
I20260811 22:14:24.727752 23649 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:24.746766 23649 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:24.750370 23649 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:24.750602 23649 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:24.758481 23649 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.189:42943
I20260811 22:14:24.758790 23726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.189:42943 every 8 connection(s)
I20260811 22:14:24.759675 23649 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/data/info.pb
I20260811 22:14:24.762565 23727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:24.766535 23727 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:14:24.769572 23642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } attempt: 3
I20260811 22:14:24.769929 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 23649
I20260811 22:14:24.770092 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-1/wal/instance
I20260811 22:14:24.770457 28862 external_mini_cluster.cc:1428] Setting key 03b0e932e73ca7e684d6ddba8827bdd9
Aug 11 22:14:24 dist-test-slave-vxj2 krb5kdc[23553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for kudu/127.28.47.189@KRBTEST.COM
Aug 11 22:14:24 dist-test-slave-vxj2 krb5kdc[23553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.189@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.28.47.188@KRBTEST.COM; defaulting to no policy
I20260811 22:14:24.783730 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.770024 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:42943 (local address 127.0.0.1:57026)
0811 22:14:24.770348 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.770361 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:24.770447 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:24.771782 (+ 1335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:24.771785 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:24.772007 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:24.772135 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:24.772142 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.774052 (+ 1910us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.774056 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:24.774857 (+ 801us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:24.774872 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.775596 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.775601 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:24.775735 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:24.776401 (+ 666us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:24.776445 (+ 44us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:24.778264 (+ 1819us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:24.781291 (+ 3027us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:24.781304 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:24.781307 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:24.781567 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:24.781952 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:24.781958 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:24.781962 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:24.782067 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:24.783103 (+ 1036us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:24.783106 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:24.783210 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 22:14:24.783357 (+ 147us) client_negotiation.cc:240] Negotiation successful
0811 22:14:24.783546 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.28.47.188@KRBTEST.COM" created.
I20260811 22:14:24.784678 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.768747 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60388 (local address 127.28.47.190:34757)
0811 22:14:24.768976 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:24.768980 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:24.770224 (+ 1244us) server_negotiation.cc:407] Connection header received
0811 22:14:24.771562 (+ 1338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:24.771568 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:24.771640 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:24.771761 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:24.772884 (+ 1123us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.773497 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.774930 (+ 1433us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.775420 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.775550 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:24.779297 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:24.779304 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:24.779308 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:24.779336 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:24.781423 (+ 2087us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:24.782022 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:24.782025 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:24.782027 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:24.782132 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:24.782485 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:24.782489 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:24.782490 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:24.782730 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:24.782857 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:24.784377 (+ 1520us) server_negotiation.cc:299] Negotiation successful
0811 22:14:24.784531 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260811 22:14:24.785290 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.770329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57026 (local address 127.28.47.189:42943)
0811 22:14:24.770695 (+ 366us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:24.770706 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:24.770746 (+ 40us) server_negotiation.cc:407] Connection header received
0811 22:14:24.771294 (+ 548us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:24.771324 (+ 30us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:24.771597 (+ 273us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:24.771789 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:24.772254 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.773912 (+ 1658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.774986 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.775418 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.776018 (+ 600us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:24.778448 (+ 2430us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:24.778455 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:24.778468 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:24.778493 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:24.781053 (+ 2560us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:24.781702 (+ 649us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:24.781708 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:24.781709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:24.781844 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:24.782170 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:24.782177 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:24.782182 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:24.782979 (+ 797us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:24.783097 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:24.783486 (+ 389us) server_negotiation.cc:299] Negotiation successful
0811 22:14:24.783934 (+ 448us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":40,"server-negotiator.queue_time_us":275,"thread_start_us":181,"threads_started":1}
I20260811 22:14:24.785555 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.769224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:34757 (local address 127.0.0.1:60388)
0811 22:14:24.770001 (+ 777us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.770102 (+ 101us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:24.771291 (+ 1189us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:24.771982 (+ 691us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:24.771991 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:24.772525 (+ 534us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:24.772723 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:24.772735 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.773624 (+ 889us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.773629 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:24.774774 (+ 1145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:24.774787 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.775629 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.775633 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:24.775816 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:24.776634 (+ 818us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:24.776659 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:24.779109 (+ 2450us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:24.781553 (+ 2444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:24.781563 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:24.781576 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:24.781880 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:24.782281 (+ 401us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:24.782283 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:24.782285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:24.782375 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:24.783018 (+ 643us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:24.783025 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:24.783248 (+ 223us) client_negotiation.cc:769] Sending connection context
0811 22:14:24.783453 (+ 205us) client_negotiation.cc:240] Negotiation successful
0811 22:14:24.785368 (+ 1915us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":393,"mutex_wait_us":1533,"thread_start_us":149,"threads_started":1}
I20260811 22:14:24.787647 23727 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:14:24.788057 23642 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:14:24.788834 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.788535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:32960)
0811 22:14:24.788667 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.788742 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260811 22:14:24.789851 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.789514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:32968)
0811 22:14:24.789705 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.789774 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112}
W20260811 22:14:24.790055 23642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:46239: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Aug 11 22:14:24 dist-test-slave-vxj2 krb5kdc[23553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.189@KRBTEST.COM for kudu/127.28.47.189@KRBTEST.COM
I20260811 22:14:24.797760 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.788237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:42943 (local address 127.0.0.1:57042)
0811 22:14:24.788428 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.788445 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:24.788550 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:24.788981 (+ 431us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:24.788984 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:24.789255 (+ 271us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:24.789400 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:24.789408 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.790457 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.790464 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:24.791066 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:24.791074 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.791599 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.791603 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:24.791712 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:24.792290 (+ 578us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:24.792303 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:24.793962 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:24.796049 (+ 2087us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:24.796052 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:24.796054 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:24.796257 (+ 203us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:24.796567 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:24.796570 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:24.796572 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:24.796680 (+ 108us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:24.797246 (+ 566us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:24.797251 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:24.797355 (+ 104us) client_negotiation.cc:769] Sending connection context
0811 22:14:24.797490 (+ 135us) client_negotiation.cc:240] Negotiation successful
0811 22:14:24.797622 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108}
I20260811 22:14:24.798135 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.788248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57042 (local address 127.28.47.189:42943)
0811 22:14:24.788430 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:24.788437 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:24.788496 (+ 59us) server_negotiation.cc:407] Connection header received
0811 22:14:24.788656 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:24.788659 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:24.788707 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:24.788787 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:24.789535 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.790337 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.791199 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:24.791472 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:24.791599 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:24.794129 (+ 2530us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:24.794136 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:24.794141 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:24.794177 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:24.795926 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:24.796359 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:24.796378 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:24.796380 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:24.796463 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:24.796817 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:24.796821 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:24.796822 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:24.797046 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:24.797137 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:24.797605 (+ 468us) server_negotiation.cc:299] Negotiation successful
0811 22:14:24.797791 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":110}
I20260811 22:14:24.798935 23727 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:14:24.799821 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.799488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:32980)
0811 22:14:24.799644 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.799732 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:14:24.800518 23673 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
I20260811 22:14:24.801554 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.801321 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:32996)
0811 22:14:24.801447 (+ 126us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.801502 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
W20260811 22:14:24.801755 23727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:46239: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.28.47.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.188.keytab.
Entry for principal kudu/127.28.47.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.188.keytab.
I20260811 22:14:24.805917 28862 external_mini_cluster.cc:1326] Running /tmp/dist-test-tasktpwotM/build/debug/bin/kudu
/tmp/dist-test-tasktpwotM/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.188:46239
--webserver_interface=127.28.47.188
--webserver_port=0
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.188.keytab
--principal=kudu/127.28.47.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.28.47.148:34873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.28.47.190:34757,127.28.47.189:42943,127.28.47.188:46239
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260811 22:14:24.821265 23642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } attempt: 1
I20260811 22:14:24.822289 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.821959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:33006)
0811 22:14:24.822142 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.822206 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":77}
I20260811 22:14:24.823148 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.822923 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:33016)
0811 22:14:24.823041 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.823089 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260811 22:14:24.823405 23642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:46239: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
I20260811 22:14:24.851117 23727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } attempt: 1
I20260811 22:14:24.852305 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.851835 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:33020)
0811 22:14:24.852048 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.852139 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":78}
I20260811 22:14:24.853238 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.852907 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:33026)
0811 22:14:24.853056 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.853123 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260811 22:14:24.853399 23727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:46239: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
I20260811 22:14:24.898669 23642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } attempt: 2
I20260811 22:14:24.899632 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.899333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:33036)
0811 22:14:24.899489 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.899553 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260811 22:14:24.900540 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.900253 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:33052)
0811 22:14:24.900385 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.900456 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260811 22:14:24.900735 23642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:46239: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
I20260811 22:14:24.921582 23727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } attempt: 2
I20260811 22:14:24.922612 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.922146 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:33056)
0811 22:14:24.922323 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.922484 (+ 161us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260811 22:14:24.923512 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:24.923262 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:33070)
0811 22:14:24.923380 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:24.923445 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260811 22:14:24.923676 23727 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:46239: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
W20260811 22:14:24.927254 23734 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260811 22:14:24.927561 23734 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260811 22:14:24.927587 23734 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260811 22:14:24.927627 23734 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260811 22:14:24.931874 23734 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260811 22:14:24.931988 23734 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260811 22:14:24.932011 23734 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260811 22:14:24.932030 23734 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260811 22:14:24.932046 23734 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260811 22:14:24.938457 23734 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.28.47.148:34873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.28.47.190:34757,127.28.47.189:42943,127.28.47.188:46239
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/krb5kdc/kudu_127.28.47.188.keytab
--principal=kudu/127.28.47.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.28.47.188:46239
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.28.47.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23734
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 11 Aug 2026 21:38:32 UTC on ccycloud.quasar-aqvrvy.root.comops.site
build id 4581604
I20260811 22:14:24.939663 23734 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260811 22:14:24.940701 23734 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260811 22:14:24.949582 23745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:24.949572 23743 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260811 22:14:24.949599 23740 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260811 22:14:24.949664 23734 server_base.cc:1034] running on GCE node
I20260811 22:14:24.950424 23734 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260811 22:14:24.951684 23734 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260811 22:14:24.952939 23734 hybrid_clock.cc:648] HybridClock initialized: now 1786486464952934 us; error 45 us; skew 500 ppm
Aug 11 22:14:24 dist-test-slave-vxj2 krb5kdc[23553](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260811 22:14:24.956112 23734 init.cc:377] Logged in from keytab as kudu/127.28.47.188@KRBTEST.COM (short username kudu)
I20260811 22:14:24.957137 23734 webserver.cc:460] Webserver started at http://127.28.47.188:38501/ using document root <none> and password file <none>
I20260811 22:14:24.957837 23734 fs_manager.cc:359] Metadata directory not provided
I20260811 22:14:24.957952 23734 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260811 22:14:24.958216 23734 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260811 22:14:24.960530 23734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/data/instance:
uuid: "b7a1b5d191d9467d8cfe044d7a8040f1"
format_stamp: "Formatted at 2026-08-11 22:14:24 on dist-test-slave-vxj2"
server_key: "ac34f1560b92f9aea34fa6c905f96838"
server_key_iv: "21169119e9e1b11727a78ce6dd8fdc31"
server_key_version: "encryptionkey@0"
I20260811 22:14:24.961198 23734 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/wal/instance:
uuid: "b7a1b5d191d9467d8cfe044d7a8040f1"
format_stamp: "Formatted at 2026-08-11 22:14:24 on dist-test-slave-vxj2"
server_key: "ac34f1560b92f9aea34fa6c905f96838"
server_key_iv: "21169119e9e1b11727a78ce6dd8fdc31"
server_key_version: "encryptionkey@0"
I20260811 22:14:24.965569 23734 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260811 22:14:24.968370 23754 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260811 22:14:24.969741 23734 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260811 22:14:24.969920 23734 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/data,/tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/wal
uuid: "b7a1b5d191d9467d8cfe044d7a8040f1"
format_stamp: "Formatted at 2026-08-11 22:14:24 on dist-test-slave-vxj2"
server_key: "ac34f1560b92f9aea34fa6c905f96838"
server_key_iv: "21169119e9e1b11727a78ce6dd8fdc31"
server_key_version: "encryptionkey@0"
I20260811 22:14:24.970059 23734 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260811 22:14:24.989604 23734 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260811 22:14:24.993389 23734 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260811 22:14:24.993619 23734 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260811 22:14:24.999993 23642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } attempt: 3
I20260811 22:14:25.000964 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.000591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:33084)
0811 22:14:25.000804 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:25.000879 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260811 22:14:25.001901 23734 rpc_server.cc:307] RPC server started. Bound to: 127.28.47.188:46239
I20260811 22:14:25.002009 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.001713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:33098)
0811 22:14:25.001863 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:25.001929 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
W20260811 22:14:25.002235 23642 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.28.47.188:46239: Network error: Client connection negotiation failed: client connection to 127.28.47.188:46239: connect: Connection refused (error 111)
I20260811 22:14:25.002384 23816 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.28.47.188:46239 every 8 connection(s)
I20260811 22:14:25.003170 23734 server_base.cc:1166] Dumped server information to /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/data/info.pb
I20260811 22:14:25.003959 28862 external_mini_cluster.cc:1388] Started /tmp/dist-test-tasktpwotM/build/debug/bin/kudu as pid 23734
I20260811 22:14:25.004120 28862 external_mini_cluster.cc:1402] Reading /tmp/dist-test-tasktpwotM/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786486147647525-28862-0/minicluster-data/master-2/wal/instance
I20260811 22:14:25.004395 28862 external_mini_cluster.cc:1428] Setting key 861edb7c21b8d38489658ce32fd34212
I20260811 22:14:25.007195 23817 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260811 22:14:25.012326 23817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:14:25.014834 23727 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } attempt: 3
Aug 11 22:14:25 dist-test-slave-vxj2 krb5kdc[23553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.188@KRBTEST.COM
Aug 11 22:14:25 dist-test-slave-vxj2 krb5kdc[23553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.189@KRBTEST.COM for kudu/127.28.47.188@KRBTEST.COM
Aug 11 22:14:25 dist-test-slave-vxj2 krb5kdc[23553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.188@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:14:25.022984 23820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.005880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33112 (local address 127.28.47.188:46239)
0811 22:14:25.006327 (+ 447us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.006348 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.006395 (+ 47us) server_negotiation.cc:407] Connection header received
0811 22:14:25.007337 (+ 942us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.007362 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.007828 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.008223 (+ 395us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.009103 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.010737 (+ 1634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.011700 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.012412 (+ 712us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.012584 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.015957 (+ 3373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:25.015963 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:25.015975 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:25.016007 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:25.018873 (+ 2866us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.019709 (+ 836us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.019728 (+ 19us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.019736 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.019912 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.020209 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.020213 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.020215 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.020833 (+ 618us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:25.021076 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:25.021548 (+ 472us) server_negotiation.cc:299] Negotiation successful
0811 22:14:25.021909 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":288,"thread_start_us":104,"threads_started":1}
I20260811 22:14:25.026876 23729 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.015274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:33120)
0811 22:14:25.015531 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:25.015545 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:25.015633 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:25.016173 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:25.016177 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:25.016413 (+ 236us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:25.016538 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:25.016546 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.017429 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.017437 (+ 8us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:25.018609 (+ 1172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:25.018621 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.019158 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.019162 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:25.019247 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.019661 (+ 414us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:25.019681 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:25.022051 (+ 2370us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:25.024822 (+ 2771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:25.024827 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:25.024829 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:25.025160 (+ 331us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:25.025545 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:25.025549 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:25.025551 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:25.025676 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:25.026275 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:25.026279 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:25.026400 (+ 121us) client_negotiation.cc:769] Sending connection context
0811 22:14:25.026530 (+ 130us) client_negotiation.cc:240] Negotiation successful
0811 22:14:25.026705 (+ 175us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177}
I20260811 22:14:25.027040 23645 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.014583 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60396 (local address 127.28.47.190:34757)
0811 22:14:25.014775 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.014799 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.015223 (+ 424us) server_negotiation.cc:407] Connection header received
0811 22:14:25.015504 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.015508 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.015573 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.015655 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.016645 (+ 990us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.017322 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.018631 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.019053 (+ 422us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.019176 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.022781 (+ 3605us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:25.022788 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:25.022792 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:25.022836 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:25.024864 (+ 2028us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.025452 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.025456 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.025458 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.025557 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.025918 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.025921 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.025923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.026194 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:25.026273 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:25.026755 (+ 482us) server_negotiation.cc:299] Negotiation successful
0811 22:14:25.026877 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260811 22:14:25.027592 23823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.015582 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33120 (local address 127.28.47.188:46239)
0811 22:14:25.015906 (+ 324us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.015915 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.015928 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:14:25.015960 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.015965 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.016007 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.016104 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.016674 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.017304 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.018752 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.019023 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.019138 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.022227 (+ 3089us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:25.022230 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:25.022232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:25.022257 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:25.024671 (+ 2414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.025336 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.025340 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.025342 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.025451 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.025821 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.025824 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.025826 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.026083 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:25.026244 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:25.027233 (+ 989us) server_negotiation.cc:299] Negotiation successful
0811 22:14:25.027363 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":164,"threads_started":1}
I20260811 22:14:25.027626 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.014496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.190:34757 (local address 127.0.0.1:60396)
0811 22:14:25.015069 (+ 573us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:25.015133 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:25.015344 (+ 211us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:25.015826 (+ 482us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:25.015832 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:25.016325 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:25.016491 (+ 166us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:25.016502 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.017459 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.017462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:25.018467 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:25.018480 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.019334 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.019341 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:25.019481 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.019980 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:25.019996 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:25.022523 (+ 2527us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:25.024986 (+ 2463us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:25.024997 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:25.025003 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:25.025354 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:25.025678 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:25.025681 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:25.025683 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:25.025797 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:25.026360 (+ 563us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:25.026367 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:25.026508 (+ 141us) client_negotiation.cc:769] Sending connection context
0811 22:14:25.027158 (+ 650us) client_negotiation.cc:240] Negotiation successful
0811 22:14:25.027451 (+ 293us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"mutex_wait_us":78,"thread_start_us":208,"threads_started":1}
I20260811 22:14:25.028861 23817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } has no permanent_uuid. Determining permanent_uuid...
I20260811 22:14:25.031255 23727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09: Bootstrap starting.
Aug 11 22:14:25 dist-test-slave-vxj2 krb5kdc[23553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.190@KRBTEST.COM
I20260811 22:14:25.034631 23727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09: Neither blocks nor log segments found. Creating new log.
Aug 11 22:14:25 dist-test-slave-vxj2 krb5kdc[23553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.188@KRBTEST.COM for kudu/127.28.47.189@KRBTEST.COM
I20260811 22:14:25.035668 23727 log.cc:824] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:25.036552 23824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.025867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60406 (local address 127.28.47.190:34757)
0811 22:14:25.026240 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.026264 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.026280 (+ 16us) server_negotiation.cc:407] Connection header received
0811 22:14:25.026319 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.026323 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.026374 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.026499 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.027399 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.028236 (+ 837us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.029336 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.029686 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.030285 (+ 599us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.032921 (+ 2636us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:25.032924 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:25.032926 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:25.032952 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:25.034590 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.035171 (+ 581us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.035175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.035177 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.035278 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.035578 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.035582 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.035583 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.035774 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:25.036053 (+ 279us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:25.036214 (+ 161us) server_negotiation.cc:299] Negotiation successful
0811 22:14:25.036342 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":271,"thread_start_us":85,"threads_started":1}
I20260811 22:14:25.039173 23727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09: No bootstrap required, opened a new log
I20260811 22:14:25.039844 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.029457 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.189:42943 (local address 127.0.0.1:57048)
0811 22:14:25.029727 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:25.029745 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:25.029827 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:25.030115 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:25.030120 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:25.030329 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:25.030466 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:25.030475 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.031460 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.031464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:25.032232 (+ 768us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:25.032244 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.032975 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.032979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:25.033072 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.033798 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:25.034031 (+ 233us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:25.035583 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:25.038031 (+ 2448us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:25.038034 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:25.038036 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:25.038303 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:25.038630 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:25.038634 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:25.038636 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:25.038753 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:25.039278 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:25.039281 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:25.039392 (+ 111us) client_negotiation.cc:769] Sending connection context
0811 22:14:25.039509 (+ 117us) client_negotiation.cc:240] Negotiation successful
0811 22:14:25.039672 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":192}
I20260811 22:14:25.040504 23730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.029749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57048 (local address 127.28.47.189:42943)
0811 22:14:25.029879 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.029883 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.029896 (+ 13us) server_negotiation.cc:407] Connection header received
0811 22:14:25.029930 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.029933 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.029977 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.030067 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.030583 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.031323 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.032374 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.032731 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.032816 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.035757 (+ 2941us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:25.035761 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:25.035764 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:25.035795 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:25.037893 (+ 2098us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.038412 (+ 519us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.038416 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.038418 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.038513 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.038864 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.038867 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.038869 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.039162 (+ 293us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:25.039282 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:25.040177 (+ 895us) server_negotiation.cc:299] Negotiation successful
0811 22:14:25.040322 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260811 22:14:25.041251 23817 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } has no permanent_uuid. Determining permanent_uuid...
Aug 11 22:14:25 dist-test-slave-vxj2 krb5kdc[23553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.28.47.189@KRBTEST.COM
I20260811 22:14:25.044190 23727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } }
I20260811 22:14:25.044481 23727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7fa8399bdfb340dd98517a0baf26bb09, State: Initialized, Role: FOLLOWER
I20260811 22:14:25.044868 23727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } }
I20260811 22:14:25.045511 23827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } } }
I20260811 22:14:25.045679 23827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:14:25.046655 23727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09 [sys.catalog]: configured and running, proceeding with master startup.
Aug 11 22:14:25 dist-test-slave-vxj2 krb5kdc[23553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.188@KRBTEST.COM for kudu/127.28.47.188@KRBTEST.COM
I20260811 22:14:25.050174 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.037837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57052 (local address 127.28.47.189:42943)
0811 22:14:25.038502 (+ 665us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.038508 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.038518 (+ 10us) server_negotiation.cc:407] Connection header received
0811 22:14:25.038548 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.038550 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.038620 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.038714 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.039508 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.040315 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.041407 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.041814 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.043651 (+ 1837us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.045319 (+ 1668us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:25.045324 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:25.045329 (+ 5us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:25.045359 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:25.048068 (+ 2709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.048691 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.048696 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.048698 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.048827 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.049182 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.049188 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.049191 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.049442 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:25.049684 (+ 242us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:25.049843 (+ 159us) server_negotiation.cc:299] Negotiation successful
0811 22:14:25.049973 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":601,"thread_start_us":131,"threads_started":1}
I20260811 22:14:25.051285 23821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.041852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:33126)
0811 22:14:25.042093 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:25.042106 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:25.042202 (+ 96us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:25.042498 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:25.042502 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:25.042700 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:25.042811 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:25.042821 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.043827 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.043830 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:25.044676 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:25.044687 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.045442 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.045445 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:25.045514 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.046081 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:25.046096 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:25.047496 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:25.049646 (+ 2150us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:25.049648 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:25.049651 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:25.049906 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:25.050178 (+ 272us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:25.050180 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:25.050182 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:25.050259 (+ 77us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:25.050741 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:25.050746 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:25.050843 (+ 97us) client_negotiation.cc:769] Sending connection context
0811 22:14:25.050959 (+ 116us) client_negotiation.cc:240] Negotiation successful
0811 22:14:25.051121 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195}
I20260811 22:14:25.051697 23823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.041923 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33126 (local address 127.28.47.188:46239)
0811 22:14:25.042033 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.042037 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.042180 (+ 143us) server_negotiation.cc:407] Connection header received
0811 22:14:25.042319 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.042322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.042370 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.042453 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.042943 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.043724 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.044819 (+ 1095us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.045227 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.045332 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.047663 (+ 2331us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:25.047670 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:25.047676 (+ 6us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:25.047702 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:25.049532 (+ 1830us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.050005 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.050008 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.050010 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.050087 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.050343 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.050346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.050347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.050606 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:25.050704 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:25.051465 (+ 761us) server_negotiation.cc:299] Negotiation successful
0811 22:14:25.051575 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260811 22:14:25.053829 28862 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260811 22:14:25.050994 23649 master_runner.cc:186] Error getting master registration for 127.28.47.190:34757: OK, instance_id { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" instance_seqno: 1786486464521891 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:14:25.054987 23817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1: Bootstrap starting.
I20260811 22:14:25.055090 23649 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 22:14:25.055372 23838 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:14:25.055449 23838 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:14:25.055064 23616 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "1a44720fd92443ad9a407675799fd9a9"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60406
I20260811 22:14:25.057608 23817 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:25.058559 23817 log.cc:824] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:25.061024 23817 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1: No bootstrap required, opened a new log
I20260811 22:14:25.065748 23817 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } }
I20260811 22:14:25.066143 23817 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7a1b5d191d9467d8cfe044d7a8040f1, State: Initialized, Role: FOLLOWER
I20260811 22:14:25.066582 23817 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } }
I20260811 22:14:25.067258 23841 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } } }
I20260811 22:14:25.067441 23841 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:14:25.068291 23817 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1 [sys.catalog]: configured and running, proceeding with master startup.
W20260811 22:14:25.074599 23855 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:14:25.074697 23855 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:14:25.072914 23734 master_runner.cc:186] Error getting master registration for 127.28.47.190:34757: OK, instance_id { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" instance_seqno: 1786486464521891 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260811 22:14:25.076457 23734 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260811 22:14:25.166430 23642 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } attempt: 4
Aug 11 22:14:25 dist-test-slave-vxj2 krb5kdc[23553](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786486464, etypes {rep=17 tkt=17 ses=17}, kudu/127.28.47.190@KRBTEST.COM for kudu/127.28.47.188@KRBTEST.COM
I20260811 22:14:25.178112 23646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.167140 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.28.47.188:46239 (local address 127.0.0.1:33134)
0811 22:14:25.167523 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0811 22:14:25.167539 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0811 22:14:25.167644 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0811 22:14:25.168279 (+ 635us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0811 22:14:25.168282 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0811 22:14:25.168575 (+ 293us) client_negotiation.cc:189] Negotiated authn=SASL
0811 22:14:25.168700 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:25.168708 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.169901 (+ 1193us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.169905 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:25.170806 (+ 901us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0811 22:14:25.170819 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.171399 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.171403 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0811 22:14:25.171500 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.172123 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0811 22:14:25.172140 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0811 22:14:25.174001 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0811 22:14:25.176246 (+ 2245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:25.176250 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:25.176252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:25.176535 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:25.176890 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0811 22:14:25.176893 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0811 22:14:25.176895 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0811 22:14:25.177012 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0811 22:14:25.177519 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0811 22:14:25.177523 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0811 22:14:25.177640 (+ 117us) client_negotiation.cc:769] Sending connection context
0811 22:14:25.177755 (+ 115us) client_negotiation.cc:240] Negotiation successful
0811 22:14:25.177924 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":279}
I20260811 22:14:25.178136 23823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.167816 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33134 (local address 127.28.47.188:46239)
0811 22:14:25.167944 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.167949 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.167963 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:14:25.168011 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.168014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.168059 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.168135 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.168923 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.169756 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.170943 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.171254 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.171378 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.174175 (+ 2797us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:25.174179 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:25.174182 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:25.174217 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:25.176093 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.176667 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.176671 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.176673 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.176779 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.177113 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.177117 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.177119 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.177404 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:25.177513 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:25.177828 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 22:14:25.177981 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260811 22:14:25.182657 23642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9: Bootstrap starting.
I20260811 22:14:25.185945 23642 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9: Neither blocks nor log segments found. Creating new log.
I20260811 22:14:25.186976 23642 log.cc:824] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9: Log is configured to *not* fsync() on all Append() calls
I20260811 22:14:25.189596 23642 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9: No bootstrap required, opened a new log
I20260811 22:14:25.192791 23642 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } }
I20260811 22:14:25.193037 23642 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1a44720fd92443ad9a407675799fd9a9, State: Initialized, Role: FOLLOWER
I20260811 22:14:25.193440 23642 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } }
I20260811 22:14:25.194058 23858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } } }
I20260811 22:14:25.194206 23858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:14:25.194978 23642 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [sys.catalog]: configured and running, proceeding with master startup.
I20260811 22:14:25.197446 23616 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260811 22:14:25.198153 23616 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:25.199888 23616 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } }
I20260811 22:14:25.201848 23698 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a44720fd92443ad9a407675799fd9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "7fa8399bdfb340dd98517a0baf26bb09"
I20260811 22:14:25.202167 23698 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:25.202209 23785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "1a44720fd92443ad9a407675799fd9a9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1"
I20260811 22:14:25.202467 23785 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1 [term 0 FOLLOWER]: Advancing to term 1
I20260811 22:14:25.203862 23785 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a44720fd92443ad9a407675799fd9a9 in term 1.
I20260811 22:14:25.204120 23698 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1a44720fd92443ad9a407675799fd9a9 in term 1.
I20260811 22:14:25.205559 23591 leader_election.cc:302] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1a44720fd92443ad9a407675799fd9a9, b7a1b5d191d9467d8cfe044d7a8040f1; no voters:
I20260811 22:14:25.205909 23858 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [term 1 FOLLOWER]: Leader election won for term 1
I20260811 22:14:25.206157 23858 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [term 1 LEADER]: Becoming Leader. State: Replica: 1a44720fd92443ad9a407675799fd9a9, State: Running, Role: LEADER
I20260811 22:14:25.206537 23858 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } }
I20260811 22:14:25.208549 23858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a44720fd92443ad9a407675799fd9a9. Latest consensus state: current_term: 1 leader_uuid: "1a44720fd92443ad9a407675799fd9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } } }
I20260811 22:14:25.208666 23858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:25.209260 23616 leader_election.cc:288] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [CANDIDATE]: Term 1 election: Requested vote from peers 7fa8399bdfb340dd98517a0baf26bb09 (127.28.47.189:42943), b7a1b5d191d9467d8cfe044d7a8040f1 (127.28.47.188:46239)
I20260811 22:14:25.210474 23567 master_runner.cc:263] No leader master found from master 1a44720fd92443ad9a407675799fd9a9
I20260811 22:14:25.210549 23567 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260811 22:14:25.212870 23870 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260811 22:14:25.212973 23870 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260811 22:14:25.213279 23874 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260811 22:14:25.214301 23874 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260811 22:14:25.234397 23785 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1 [term 1 FOLLOWER]: Refusing update from remote peer 1a44720fd92443ad9a407675799fd9a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:25.235075 23824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.211932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60414 (local address 127.28.47.190:34757)
0811 22:14:25.212057 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.212061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.217503 (+ 5442us) server_negotiation.cc:407] Connection header received
0811 22:14:25.217668 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.217672 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.217731 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.217825 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.222369 (+ 4544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.223537 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.224738 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.225100 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.225222 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.229811 (+ 4589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:25.229816 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:25.229818 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:25.229852 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:25.231832 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.232376 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.232380 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.232382 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.232474 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.232911 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.232915 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.232917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.233175 (+ 258us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:25.233320 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:25.234752 (+ 1432us) server_negotiation.cc:299] Negotiation successful
0811 22:14:25.234904 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260811 22:14:25.235263 23871 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260811 22:14:25.240062 23841 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a44720fd92443ad9a407675799fd9a9. Latest consensus state: current_term: 1 leader_uuid: "1a44720fd92443ad9a407675799fd9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } } }
I20260811 22:14:25.240260 23841 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:14:25.242007 23698 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09 [term 1 FOLLOWER]: Refusing update from remote peer 1a44720fd92443ad9a407675799fd9a9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260811 22:14:25.243046 23871 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260811 22:14:25.245090 23760 tablet.cc:2369] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:14:25.248010 23871 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a44720fd92443ad9a407675799fd9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } } }
I20260811 22:14:25.248176 23871 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:25.248351 23827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1a44720fd92443ad9a407675799fd9a9. Latest consensus state: current_term: 1 leader_uuid: "1a44720fd92443ad9a407675799fd9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } } }
I20260811 22:14:25.248468 23827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:14:25.249555 23858 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "1a44720fd92443ad9a407675799fd9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } } }
I20260811 22:14:25.249679 23858 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9 [sys.catalog]: This master's current role is: LEADER
I20260811 22:14:25.250085 23841 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a44720fd92443ad9a407675799fd9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } } }
I20260811 22:14:25.250211 23841 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:14:25.250837 23827 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "1a44720fd92443ad9a407675799fd9a9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1a44720fd92443ad9a407675799fd9a9" member_type: VOTER last_known_addr { host: "127.28.47.190" port: 34757 } } peers { permanent_uuid: "7fa8399bdfb340dd98517a0baf26bb09" member_type: VOTER last_known_addr { host: "127.28.47.189" port: 42943 } } peers { permanent_uuid: "b7a1b5d191d9467d8cfe044d7a8040f1" member_type: VOTER last_known_addr { host: "127.28.47.188" port: 46239 } } }
I20260811 22:14:25.250962 23827 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09 [sys.catalog]: This master's current role is: FOLLOWER
I20260811 22:14:25.251312 23884 mvcc.cc:205] Tried to move back new op lower bound from 7317448561576091648 to 7317448561489752064. Current Snapshot: MvccSnapshot[applied={T|T < 7317448561576091648}]
W20260811 22:14:25.252384 23675 tablet.cc:2369] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260811 22:14:25.253348 23874 catalog_manager.cc:1347] Generated new cluster ID: 4e5e196e581f466f8807959a63f53cb3
I20260811 22:14:25.253425 23874 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260811 22:14:25.254506 23887 mvcc.cc:205] Tried to move back new op lower bound from 7317448561576091648 to 7317448561489752064. Current Snapshot: MvccSnapshot[applied={T|T < 7317448561576091648}]
I20260811 22:14:25.257893 23826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.225872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57068 (local address 127.28.47.189:42943)
0811 22:14:25.226005 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.226010 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.229465 (+ 3455us) server_negotiation.cc:407] Connection header received
0811 22:14:25.229644 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.229648 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.229712 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.229770 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.230725 (+ 955us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.231607 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.241277 (+ 9670us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.241656 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.241742 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.253970 (+ 12228us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:25.253974 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:25.253977 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:25.254015 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:25.255926 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.256553 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.256556 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.256558 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.256640 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.256921 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.256924 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.256926 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.257124 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:25.257262 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:25.257577 (+ 315us) server_negotiation.cc:299] Negotiation successful
0811 22:14:25.257720 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260811 22:14:25.260070 23823 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:25.212207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33150 (local address 127.28.47.188:46239)
0811 22:14:25.212314 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.212318 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.225688 (+ 13370us) server_negotiation.cc:407] Connection header received
0811 22:14:25.225941 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.225945 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.226005 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.226065 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.227945 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.228664 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.233684 (+ 5020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:25.233997 (+ 313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:25.234080 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:25.238639 (+ 4559us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0811 22:14:25.238643 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0811 22:14:25.238646 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0811 22:14:25.238680 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0811 22:14:25.240701 (+ 2021us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.254761 (+ 14060us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.254766 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.254781 (+ 15us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.254905 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0811 22:14:25.255399 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0811 22:14:25.255403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0811 22:14:25.255405 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0811 22:14:25.255685 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0811 22:14:25.255862 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:25.259763 (+ 3901us) server_negotiation.cc:299] Negotiation successful
0811 22:14:25.259912 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260811 22:14:25.263459 23889 mvcc.cc:205] Tried to move back new op lower bound from 7317448561576091648 to 7317448561489752064. Current Snapshot: MvccSnapshot[applied={T|T < 7317448561576091648}]
I20260811 22:14:25.298491 23874 catalog_manager.cc:1370] Generated new certificate authority record
I20260811 22:14:25.299913 23874 catalog_manager.cc:1504] Loading token signing keys...
I20260811 22:14:25.310876 23874 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1a44720fd92443ad9a407675799fd9a9: Generated new TSK 0
I20260811 22:14:25.311875 23874 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260811 22:14:25.345527 23824 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:25.343870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60424 (local address 127.28.47.190:34757)
0811 22:14:25.344020 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.344025 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.344191 (+ 166us) server_negotiation.cc:407] Connection header received
0811 22:14:25.344437 (+ 246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.344440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.344507 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.344602 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.345385 (+ 783us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:60424: BlockingRecv error: recv got EOF from 127.0.0.1:60424 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 22:14:25.346444 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:25.343420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57082 (local address 127.28.47.189:42943)
0811 22:14:25.344029 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.344035 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.344680 (+ 645us) server_negotiation.cc:407] Connection header received
0811 22:14:25.344880 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.344887 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.344949 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.345009 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.346278 (+ 1269us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57082: BlockingRecv error: recv got EOF from 127.0.0.1:57082 (error 108)
Metrics: {"server-negotiator.queue_time_us":492}
W20260811 22:14:25.346465 23823 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:25.343586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33158 (local address 127.28.47.188:46239)
0811 22:14:25.343872 (+ 286us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.343879 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.344337 (+ 458us) server_negotiation.cc:407] Connection header received
0811 22:14:25.344760 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.344767 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.344828 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.344912 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.346303 (+ 1391us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33158: BlockingRecv error: recv got EOF from 127.0.0.1:33158 (error 108)
Metrics: {"server-negotiator.queue_time_us":176}
W20260811 22:14:25.350232 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:25.348997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57088 (local address 127.28.47.189:42943)
0811 22:14:25.349130 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.349133 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.349250 (+ 117us) server_negotiation.cc:407] Connection header received
0811 22:14:25.349443 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.349446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.349520 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.349586 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.350093 (+ 507us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57088: BlockingRecv error: recv got EOF from 127.0.0.1:57088 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
W20260811 22:14:25.354558 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:25.353613 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57104 (local address 127.28.47.189:42943)
0811 22:14:25.353751 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.353755 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.353772 (+ 17us) server_negotiation.cc:407] Connection header received
0811 22:14:25.353830 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.353832 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.353874 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.353981 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.354425 (+ 444us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57104: BlockingRecv error: recv got EOF from 127.0.0.1:57104 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
W20260811 22:14:25.360138 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:25.358766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57116 (local address 127.28.47.189:42943)
0811 22:14:25.358905 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.358909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.359044 (+ 135us) server_negotiation.cc:407] Connection header received
0811 22:14:25.359229 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.359232 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.359303 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.359391 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.359980 (+ 589us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57116: BlockingRecv error: recv got EOF from 127.0.0.1:57116 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260811 22:14:25.368019 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:25.366713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57124 (local address 127.28.47.189:42943)
0811 22:14:25.366943 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.366948 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.367043 (+ 95us) server_negotiation.cc:407] Connection header received
0811 22:14:25.367174 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.367178 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.367242 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.367363 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.367892 (+ 529us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57124: BlockingRecv error: recv got EOF from 127.0.0.1:57124 (error 108)
Metrics: {"server-negotiator.queue_time_us":129}
W20260811 22:14:25.379987 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:25.378595 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57132 (local address 127.28.47.189:42943)
0811 22:14:25.378738 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.378743 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.379009 (+ 266us) server_negotiation.cc:407] Connection header received
0811 22:14:25.379151 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.379155 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.379209 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.379286 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.379847 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57132: BlockingRecv error: recv got EOF from 127.0.0.1:57132 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
W20260811 22:14:25.399748 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:25.398298 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57146 (local address 127.28.47.189:42943)
0811 22:14:25.398494 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.398499 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.398629 (+ 130us) server_negotiation.cc:407] Connection header received
0811 22:14:25.398857 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.398860 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.398912 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.399029 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.399569 (+ 540us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57146: BlockingRecv error: recv got EOF from 127.0.0.1:57146 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 22:14:25.435748 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:25.434310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57156 (local address 127.28.47.189:42943)
0811 22:14:25.434506 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.434511 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.434701 (+ 190us) server_negotiation.cc:407] Connection header received
0811 22:14:25.434904 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.434910 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.434976 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.435084 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.435603 (+ 519us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57156: BlockingRecv error: recv got EOF from 127.0.0.1:57156 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260811 22:14:25.503589 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:25.502307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57168 (local address 127.28.47.189:42943)
0811 22:14:25.502473 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.502477 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.502649 (+ 172us) server_negotiation.cc:407] Connection header received
0811 22:14:25.502736 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.502739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.502830 (+ 91us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.502919 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.503466 (+ 547us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57168: BlockingRecv error: recv got EOF from 127.0.0.1:57168 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
W20260811 22:14:25.635272 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:25.634097 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57174 (local address 127.28.47.189:42943)
0811 22:14:25.634252 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.634256 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.634363 (+ 107us) server_negotiation.cc:407] Connection header received
0811 22:14:25.634471 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.634473 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.634522 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.634609 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.635122 (+ 513us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57174: BlockingRecv error: recv got EOF from 127.0.0.1:57174 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260811 22:14:25.895416 23826 negotiation.cc:337] Failed RPC negotiation. Trace:
0811 22:14:25.893985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57182 (local address 127.28.47.189:42943)
0811 22:14:25.894142 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:25.894146 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:25.894207 (+ 61us) server_negotiation.cc:407] Connection header received
0811 22:14:25.894342 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:25.894345 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:25.894397 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:25.894475 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0811 22:14:25.895263 (+ 788us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:57182: BlockingRecv error: recv got EOF from 127.0.0.1:57182 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260811 22:14:26.057044 23838 catalog_manager.cc:1259] Loaded cluster ID: 4e5e196e581f466f8807959a63f53cb3
I20260811 22:14:26.057189 23838 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09: loading cluster ID for follower catalog manager: success
I20260811 22:14:26.059095 23838 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09: acquiring CA information for follower catalog manager: success
I20260811 22:14:26.060483 23838 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 7fa8399bdfb340dd98517a0baf26bb09: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:14:26.076279 23855 catalog_manager.cc:1259] Loaded cluster ID: 4e5e196e581f466f8807959a63f53cb3
I20260811 22:14:26.076386 23855 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1: loading cluster ID for follower catalog manager: success
I20260811 22:14:26.078261 23855 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1: acquiring CA information for follower catalog manager: success
I20260811 22:14:26.079528 23855 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b7a1b5d191d9467d8cfe044d7a8040f1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260811 22:14:26.414170 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:26.409808 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57186 (local address 127.28.47.189:42943)
0811 22:14:26.410076 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:26.410081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:26.410244 (+ 163us) server_negotiation.cc:407] Connection header received
0811 22:14:26.410441 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:26.410450 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:26.410589 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:26.410719 (+ 130us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:26.411008 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:26.411565 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:26.412490 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:26.412732 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:26.412848 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:26.413264 (+ 416us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:26.413737 (+ 473us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:26.413897 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:26.413933 (+ 36us) server_negotiation.cc:299] Negotiation successful
0811 22:14:26.413999 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":107,"threads_started":1}
I20260811 22:14:26.423126 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:26.417901 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33170 (local address 127.28.47.188:46239)
0811 22:14:26.418228 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:26.418235 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:26.418256 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:14:26.418392 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:26.418397 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:26.418596 (+ 199us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:26.418758 (+ 162us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:26.418959 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:26.419961 (+ 1002us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:26.420862 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:26.421341 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:26.421466 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:26.421822 (+ 356us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:26.422583 (+ 761us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:26.422745 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:26.422786 (+ 41us) server_negotiation.cc:299] Negotiation successful
0811 22:14:26.422877 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":225,"thread_start_us":82,"threads_started":1}
I20260811 22:14:26.431995 23971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:26.427576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57194 (local address 127.28.47.189:42943)
0811 22:14:26.428045 (+ 469us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:26.428052 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:26.428073 (+ 21us) server_negotiation.cc:407] Connection header received
0811 22:14:26.428147 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:26.428152 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:26.428212 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:26.428310 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:26.428686 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:26.429467 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:26.430669 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:26.431028 (+ 359us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:26.431134 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:26.431423 (+ 289us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:26.431519 (+ 96us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:26.431589 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:26.431776 (+ 187us) server_negotiation.cc:299] Negotiation successful
0811 22:14:26.431846 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":342}
I20260811 22:14:26.433300 23978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:26.427930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33180 (local address 127.28.47.188:46239)
0811 22:14:26.428055 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:26.428060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:26.428466 (+ 406us) server_negotiation.cc:407] Connection header received
0811 22:14:26.429392 (+ 926us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:26.429395 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:26.429447 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:26.429492 (+ 45us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:26.429857 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:26.430350 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:26.431606 (+ 1256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:26.431972 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:26.432078 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:26.432604 (+ 526us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:26.432687 (+ 83us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:26.432732 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:26.432918 (+ 186us) server_negotiation.cc:299] Negotiation successful
0811 22:14:26.432980 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260811 22:14:26.433596 23985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0811 22:14:26.427740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60430 (local address 127.28.47.190:34757)
0811 22:14:26.428702 (+ 962us) server_negotiation.cc:206] Beginning negotiation
0811 22:14:26.428707 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0811 22:14:26.428721 (+ 14us) server_negotiation.cc:407] Connection header received
0811 22:14:26.428827 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0811 22:14:26.428830 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0811 22:14:26.428992 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0811 22:14:26.429117 (+ 125us) server_negotiation.cc:226] Negotiated authn=TOKEN
0811 22:14:26.429473 (+ 356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:26.430304 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:26.431580 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0811 22:14:26.432043 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0811 22:14:26.432155 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0811 22:14:26.432586 (+ 431us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0811 22:14:26.432940 (+ 354us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0811 22:14:26.433034 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0811 22:14:26.433370 (+ 336us) server_negotiation.cc:299] Negotiation successful
0811 22:14:26.433446 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":819,"thread_start_us":102,"threads_started":1}
I20260811 22:14:26.437038 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 23567
I20260811 22:14:26.448189 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 23649
I20260811 22:14:26.457705 28862 external_mini_cluster.cc:1595] Killing /tmp/dist-test-tasktpwotM/build/debug/bin/kudu with pid 23734
2026-08-11T22:14:26Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3447 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6777 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (318789 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST